Project 2 - Closing Price Stock Classifier - Version 1.0.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project 2 - Closing Price Stock Classifier - Version 1.0\n",
    "\n",
    "    Introduction\n",
    "\n",
    "Welcome to my second repository! This one is a new take on my first project, which tried to predict the closing price of a stock given today's opening price, historical data, and engineered features using an automated regression optimization tool, TPOT, to find the best regression pipeline for the dataset. That project can be found here:\n",
    "\n",
    "https://github.com/windowshopr/Project-1-Closing-Price-Stock-Regressor-Version-1.0\n",
    "\n",
    "At the end of that project, I noticed that the regressor was unable to predict the closing price very accurately because the stock I was predicting, CLVS, absolutely tanked in the last 90 days from running that model. So low that that price hadn't been seen before during the training process. I had this idea before, but now it seems even more relevant, what if instead of trying to predict the actual closing price, we just simply tried to predict a Buy or Sell signal for that day? Would it have a decent accuracy score using the exact same work as before with feature engineering, etc.? Let's find out!\n",
    "\n",
    "    DISCLAIMER: This project is merely a compilation of various Python and machine learning skills that I have picked up \n",
    "    over the past few months and years. This tool is not actually meant to be used for stock advisement, it is for academic\n",
    "    and future development purposes only.\n",
    "    \n",
    "    You've been warned. Now enjoy!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Environment and Installing the Dependencies\n",
    "\n",
    "First thing's first, if you want to run this project, you need to have the appropriate dependencies installed. Everything listed below are the versions of each dependency as of this writing. I'll explain how to install them all at the bottom of the list. Now I can't speak for other versions, but you need to be running ... Python 3.6 on Windows 10 ..., as that's what I'm running on my machine. Here are the dependencies you'll need:\n",
    "\n",
    "    DateTime==4.3\n",
    "    pandas==0.23.0\n",
    "    pandas-datareader==0.7.0\n",
    "    seaborn==0.8.1\n",
    "    matplotlib==2.1.2\n",
    "    numpy==1.14.5\n",
    "    bs4==0.0.1\n",
    "    requests==2.22.0\n",
    "    feature-selector==1.0.0\n",
    "    featuretools==0.9.1\n",
    "    dask==2.2.0\n",
    "    dask-ml==1.0.0\n",
    "    sklearn==0.0\n",
    "    TPOT==0.10.2\n",
    "    ta-lib==0.4.18\n",
    "\n",
    "Again, this is all in Python 3.6 and Windows 10. All of the other modules used in this project (like \"os\" and \"warnings\" should be included in your Python package already. If you run into something I've missed, let me know, but this should be a complete list. There are two ways to install all of these packages:\n",
    "\n",
    "\n",
    "1. Install them all at once using the Pip installer by typing the following command into your command prompt:\n",
    "\n",
    "        pip install DateTime==4.3 pandas==0.23.0 pandas-datareader==0.7.0 seaborn==0.8.1 matplotlib==2.1.2 numpy==1.14.5 bs4==0.0.1 requests==2.22.0 feature-selector==1.0.0 featuretools==0.9.1 dask==2.2.0 dask-ml==1.0.0 sklearn TPOT==0.10.2 TA-Lib\n",
    "    \n",
    "   Or\n",
    "            \n",
    "        pip3 install DateTime==4.3 pandas==0.23.0 pandas-datareader==0.7.0 seaborn==0.8.1 matplotlib==2.1.2 numpy==1.14.5 bs4==0.0.1 requests==2.22.0 feature-selector==1.0.0 featuretools==0.9.1 dask==2.2.0 dask-ml==1.0.0 sklearn TPOT==0.10.2 TA-Lib\n",
    "\n",
    "(Depending on how you have installed your Pip installer. Notice the 3 after pip)\n",
    "\n",
    "\n",
    "2. Install them all from the provided \"requirements.txt\" file by typing in the following command into your command prompt:\n",
    "\n",
    "        pip install -r requirements.txt\n",
    "        \n",
    "   Or\n",
    "           \n",
    "        pip3 install -r requirements.txt\n",
    "\n",
    "(One of those two ways should create the environment needed to run this project. Otherwise each module will need to be installed manually as needed)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References and Resources Before We Get Started\n",
    "\n",
    "Before I forget and start going down the code wormhole, I wanted to take some time and reference some various websites, other's projects and the tools that I'm using in the following code. I want to thank everyone involved in my learning process and I urge you to check them out:\n",
    "\n",
    "1. The motivation and skeleton for my project - https://gogul09.github.io/software/regression-example-boston-housing-prices\n",
    "(That article taught me more in one reading than any other)\n",
    "\n",
    "\n",
    "1. TPOT (automated machine learning) - https://epistasislab.github.io/tpot/\n",
    "2. FeatureTools (feature engineering) - https://www.featuretools.com/\n",
    "3. Feature-Selector (take a guess :D) - https://github.com/WillKoehrsen/feature-selector\n",
    "4. Dask (parallel computing) - https://docs.dask.org/en/latest/\n",
    "5. TA-Lib (technical indicators) - https://mrjbq7.github.io/ta-lib/index.html\n",
    "\n",
    "\n",
    "1. Medium (Great resource for learning. Sign up for emails based on what you read!) - https://medium.com/topic/artificial-intelligence\n",
    "2. Towards Data Science (Another awesome resource) - https://towardsdatascience.com/\n",
    "3. Machine Learning Mastery (see above) - https://machinelearningmastery.com/\n",
    "4. Stack Overflow (duh!) - https://stackoverflow.com/\n",
    "5. Upwork (in a real pinch, pay someone to teach you something) - www.upwork.com\n",
    "6. Udemy (I received great value from this course) - https://www.udemy.com/the-data-science-course-complete-data-science-bootcamp/\n",
    "\n",
    "\n",
    "Without further adu, let's get started."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Code\n",
    "\n",
    "Keep in mind, I have already run every cell in this notebook. If you want to re-run it, you are more than welcome to, however I have made references to values and numbers pertaining to this run only, like prices and accuracy values and such. My breakdowns of each step will then vary if you're looking up a different stock, so just keep that in mind if you're re-running it for the first time. Adjust as you need.\n",
    "\n",
    "ALSO, if you do want to start this script over again, make sure you click \"Kernel > Restart and Clear Output\" to make sure you're starting fresh!!!\n",
    "\n",
    "Most the project is recycled from that first one so a lot of the comments will be the same so hopefully it'll be easy to follow along."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. GET THE STOCK'S HISTORICAL DATA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Which stock ticker would you like to predict? NFLX\n",
      "Getting the historical data for:  NFLX\n"
     ]
    }
   ],
   "source": [
    "# In-notebook plotting. Take this out if running from a .py file\n",
    "%matplotlib inline\n",
    "\n",
    "# Display Future Warnings that repeat \"needlessly\" only once (for now)\n",
    "import warnings\n",
    "warnings.filterwarnings(\"once\")\n",
    "\n",
    "# Let's begin. Ask for which stock ticker you want to predict.\n",
    "# A list of stocks that are able to be run so far are:\n",
    "\n",
    "ticker_input = input('Which stock ticker would you like to predict? ') \n",
    "print('Getting the historical data for: ',ticker_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216, got 192\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            High  Low  Open  Close         Volume  Adj Close\n",
      "Date                                                        \n",
      "2002-05-23  1.24 1.15  1.16   1.20 104,790,000.00       1.20\n",
      "2002-05-24  1.23 1.20  1.21   1.21  11,104,800.00       1.21\n",
      "2002-05-28  1.23 1.16  1.21   1.16   6,609,400.00       1.16\n",
      "2002-05-29  1.16 1.09  1.16   1.10   6,757,800.00       1.10\n",
      "2002-05-30  1.11 1.07  1.11   1.07  10,154,200.00       1.07\n",
      "Done!\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\pandas_datareader\\data.py:310: ResourceWarning: unclosed <socket.socket fd=1524, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.8.8.80', 54704), raddr=('216.115.100.124', 443)>\n",
      "  session=session).read()\n"
     ]
    }
   ],
   "source": [
    "# Import first group of dependencies, and then\n",
    "# download the historical daily data from Yahoo Finance\n",
    "# using Pandas Datareader. A different source could be\n",
    "# used as I've read that getting historical data using\n",
    "# this method is temperamental (and deprecated), but works for now\n",
    "from datetime import datetime\n",
    "from pandas_datareader import data as web\n",
    "import pandas as pd\n",
    "pd.options.display.float_format = '{:,.2f}'.format\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import random\n",
    "import os\n",
    "import numpy as np\n",
    "import time\n",
    "\n",
    "# Downloading historical data as a Pandas dataframe.\n",
    "# Think of a dataframe as an Excel file, with rows\n",
    "# and columns of data, sometimes with a header row.\n",
    "ex = 'yahoo'\n",
    "start = datetime(2000, 1, 1) # Edit this to whatever you want, but the older the better\n",
    "end = datetime.now() # Today\n",
    "dataset = web.DataReader(ticker_input, ex, start, end)\n",
    "\n",
    "# Now let's look at the top 5 rows of the dataset\n",
    "print(dataset.head(5))\n",
    "\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. ADJUST THE HISTORICAL DATASET AND ADD TECHNICAL INDICATORS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We should have a Date index column, which isn't exactly a part of \n",
    "# the dataframe itself. The rest of the columns are the stock's\n",
    "# historical data per day.\n",
    "\n",
    "# Now that we have the dataframe, let's modify it for our model.\n",
    "# You'll notice the data has a Close and an Adjusted Close column.\n",
    "# We're going to use the Adj Close for various reasons, so let's\n",
    "# get rid of the Close column:\n",
    "dataset = dataset.drop(['Close'],axis=1)\n",
    "\n",
    "# Now let's shift the daily data so that for the current day,\n",
    "# we have new columns containing the previous day's OHLC data.\n",
    "dataset['PrevOpen'] = dataset['Open'].shift(1)\n",
    "dataset['PrevHigh'] = dataset['High'].shift(1)\n",
    "dataset['PrevLow'] = dataset['Low'].shift(1)\n",
    "dataset['PrevAdjClose'] = dataset['Adj Close'].shift(1)\n",
    "dataset['PrevVol'] = dataset['Volume'].shift(1)\n",
    "\n",
    "# Now that we have that data, we need to drop the High, Low and \n",
    "# Volume columns because we won't know what those values will be\n",
    "# until the trading day is over. We'll only know (roughly) where the \n",
    "# stock will open, and then all of what happened yesterday.\n",
    "dataset = dataset.drop(['High'],axis=1)\n",
    "dataset = dataset.drop(['Low'],axis=1)\n",
    "dataset = dataset.drop(['Volume'],axis=1)\n",
    "\n",
    "# Now just for fun, let's add in a couple moving averages based on the \n",
    "# opening prices. We could add in way more indicators here in the future.\n",
    "# The model can determine which ones are important. You'll see that later.\n",
    "dataset['9MA'] = dataset['Open'].rolling(window=9).mean()\n",
    "dataset['20MA'] = dataset['Open'].rolling(window=20).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This next section I've added as part of an upgrade process to my original\n",
    "# code. We're going to include the TA-Lib module, which is a great tool to\n",
    "# calculate techincal indicators based on previous stock data.\n",
    "import talib\n",
    "\n",
    "# I'm not going to get too detailed with the technical indicators, but you can\n",
    "# check out https://mrjbq7.github.io/ta-lib/doc_index.html to learn more about them,\n",
    "# but for now, let's just add a bunch (mostly momentum and volatility ones) to our dataframe and let the model decide\n",
    "# which ones are important enough to keep for training. Some would argue that too many\n",
    "# adds noise to the data, and they'd be right, but we have a step to determine feature\n",
    "# importance in a little while to combat this. We'll also deal with some of these\n",
    "# indicators correlating to one another in a second.\n",
    "\n",
    "# Overlap Studies\n",
    "# Bollinger Bands\n",
    "upperband, middleband, lowerband = talib.BBANDS(dataset['PrevAdjClose'], timeperiod=20, nbdevup=2, nbdevdn=2, matype=0)\n",
    "dataset['UpperBB'] = upperband\n",
    "dataset['MidBB'] = middleband\n",
    "dataset['LowerBB'] = lowerband\n",
    "\n",
    "# SAR - Parabolic SAR\n",
    "sar = talib.SAR(dataset['PrevHigh'], dataset['PrevLow'], acceleration=0.02, maximum=0.2)\n",
    "dataset['SAR'] = sar\n",
    "\n",
    "\n",
    "# Momentum Indicators\n",
    "# ADX - Average Directional Movement Index\n",
    "real = talib.ADX(dataset['PrevHigh'], dataset['PrevLow'], dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['ADX'] = real\n",
    "\n",
    "# APO - Absolute Price Oscillator\n",
    "real = talib.APO(dataset['PrevAdjClose'], fastperiod=12, slowperiod=26, matype=0)\n",
    "dataset['APO'] = real\n",
    "\n",
    "# CCI - Commodity Channel Index\n",
    "real = talib.CCI(dataset['PrevHigh'], dataset['PrevLow'], dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['CCI'] = real\n",
    "\n",
    "# DX - Directional Movement Index\n",
    "real = talib.DX(dataset['PrevHigh'], dataset['PrevLow'], dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['DX'] = real\n",
    "\n",
    "# MACD - Moving Average Convergence/Divergence\n",
    "macd, macdsignal, macdhist = talib.MACD(dataset['PrevAdjClose'], fastperiod=12, slowperiod=26, signalperiod=9)\n",
    "dataset['MACD'] = macd\n",
    "dataset['MACDSIGNAL'] = macdsignal\n",
    "dataset['MACDHIST'] = macdhist\n",
    "\n",
    "# MOM - Momentum\n",
    "real = talib.MOM(dataset['PrevAdjClose'], timeperiod=10)\n",
    "dataset['MOM'] = real\n",
    "\n",
    "# ROC - Rate of change : ((price/prevPrice)-1)*100\n",
    "real = talib.ROC(dataset['PrevAdjClose'], timeperiod=10)\n",
    "dataset['ROC'] = real\n",
    "\n",
    "# RSI - Relative Strength Index\n",
    "real = talib.RSI(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['RSI'] = real\n",
    "\n",
    "# STOCH - Stochastic\n",
    "slowk, slowd = talib.STOCH(dataset['PrevHigh'], dataset['PrevLow'], dataset['PrevAdjClose'], fastk_period=5, slowk_period=3, slowk_matype=0, slowd_period=3, slowd_matype=0)\n",
    "dataset['STOCHSLOWK'] = slowk\n",
    "dataset['STOCHSLOWD'] = slowd\n",
    "\n",
    "# ULTOSC - Ultimate Oscillator\n",
    "real = talib.ULTOSC(dataset['PrevHigh'], dataset['PrevLow'], dataset['PrevAdjClose'], timeperiod1=7, timeperiod2=14, timeperiod3=28)\n",
    "dataset['ULTOSC'] = real\n",
    "\n",
    "\n",
    "# Volume Indicator Functions\n",
    "# OBV - On Balance Volume\n",
    "real = talib.OBV(dataset['PrevAdjClose'], dataset['PrevVol'])\n",
    "dataset['OBV'] = real\n",
    "\n",
    "\n",
    "# Volatility Indicator Functions\n",
    "# ATR - Average True Range\n",
    "real = talib.ATR(dataset['PrevHigh'], dataset['PrevLow'], dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['ATR'] = real\n",
    "\n",
    "\n",
    "# Statistic Functions\n",
    "# BETA - Beta\n",
    "real = talib.BETA(dataset['PrevHigh'], dataset['PrevLow'], timeperiod=5)\n",
    "dataset['BETA'] = real\n",
    "\n",
    "# LINEARREG - Linear Regression\n",
    "real = talib.LINEARREG(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['LINEARREG'] = real\n",
    "\n",
    "# LINEARREG_ANGLE - Linear Regression Angle\n",
    "real = talib.LINEARREG_ANGLE(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['LINEARREG_ANGLE'] = real\n",
    "\n",
    "# LINEARREG_SLOPE - Linear Regression Slope\n",
    "real = talib.LINEARREG_SLOPE(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['LINEARREG_SLOPE'] = real\n",
    "\n",
    "# STDDEV - Standard Deviation\n",
    "real = talib.STDDEV(dataset['PrevAdjClose'], timeperiod=5, nbdev=1)\n",
    "dataset['STDDEV'] = real\n",
    "\n",
    "# TSF - Time Series Forecast\n",
    "real = talib.TSF(dataset['PrevAdjClose'], timeperiod=14)\n",
    "dataset['TSF'] = real\n",
    "\n",
    "# VAR - Variance\n",
    "#real = talib.VAR(dataset['PrevAdjClose'], timeperiod=5, nbdev=1)\n",
    "#dataset['VAR'] = real"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now let's quickly delete highly correlated features, but make sure it doesn't\n",
    "# get rid of the features we know we want to keep!\n",
    "\n",
    "# Create correlation matrix\n",
    "#corr_matrix = dataset.drop(['Open','PrevOpen','PrevHigh','PrevLow','PrevAdjClose','PrevVol','9MA','20MA','Adj Close'],axis=1).corr().abs()\n",
    "\n",
    "# Select upper triangle of correlation matrix\n",
    "#upper = corr_matrix.where(np.triu(np.ones(corr_matrix.shape), k=1).astype(np.bool))\n",
    "\n",
    "# Find index of feature columns with correlation greater than 0.98\n",
    "#to_drop = [column for column in upper.columns if any(upper[column] >= 0.98)]\n",
    "\n",
    "# Drop features \n",
    "#dataset.drop(dataset[to_drop], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            Open  Adj Close  PrevOpen  PrevHigh  PrevLow  PrevAdjClose  \\\n",
      "Date                                                                     \n",
      "2002-05-23  1.16       1.20       nan       nan      nan           nan   \n",
      "2002-05-24  1.21       1.21      1.16      1.24     1.15          1.20   \n",
      "2002-05-28  1.21       1.16      1.21      1.23     1.20          1.21   \n",
      "2002-05-29  1.16       1.10      1.21      1.23     1.16          1.16   \n",
      "2002-05-30  1.11       1.07      1.16      1.16     1.09          1.10   \n",
      "\n",
      "                  PrevVol  9MA  20MA  UpperBB ...   STOCHSLOWD  ULTOSC  \\\n",
      "Date                                          ...                        \n",
      "2002-05-23            nan  nan   nan      nan ...          nan     nan   \n",
      "2002-05-24 104,790,000.00  nan   nan      nan ...          nan     nan   \n",
      "2002-05-28  11,104,800.00  nan   nan      nan ...          nan     nan   \n",
      "2002-05-29   6,609,400.00  nan   nan      nan ...          nan     nan   \n",
      "2002-05-30   6,757,800.00  nan   nan      nan ...          nan     nan   \n",
      "\n",
      "                      OBV  ATR  BETA  LINEARREG  LINEARREG_ANGLE  \\\n",
      "Date                                                               \n",
      "2002-05-23            nan  nan   nan        nan              nan   \n",
      "2002-05-24 104,790,000.00  nan   nan        nan              nan   \n",
      "2002-05-28 115,894,800.00  nan   nan        nan              nan   \n",
      "2002-05-29 109,285,400.00  nan   nan        nan              nan   \n",
      "2002-05-30 102,527,600.00  nan   nan        nan              nan   \n",
      "\n",
      "            LINEARREG_SLOPE  STDDEV  TSF  \n",
      "Date                                      \n",
      "2002-05-23              nan     nan  nan  \n",
      "2002-05-24              nan     nan  nan  \n",
      "2002-05-28              nan     nan  nan  \n",
      "2002-05-29              nan     nan  nan  \n",
      "2002-05-30              nan     nan  nan  \n",
      "\n",
      "[5 rows x 34 columns]\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "# Phewf! Now print the new head of the dataframe and check it out so far\n",
    "print(dataset.head(5))\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reshape the dataframe to put Open and Adj Close to the far \n",
    "# right so when we export the predictions dataset, the predictions\n",
    "# column will be right next to them for easier analysis\n",
    "opencolumn = dataset['Open']\n",
    "adjclosecolumn = dataset['Adj Close']\n",
    "\n",
    "dataset = dataset.drop(['Open'],axis=1)\n",
    "dataset = dataset.drop(['Adj Close'],axis=1)\n",
    "\n",
    "dataset['Open'] = opencolumn\n",
    "dataset['Adj Close'] = adjclosecolumn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now, because we're predicting Buy and Sell signals for each day,\n",
    "# we need to make a new column that shows whether or not the stock\n",
    "# closed higher or lower than that day's open; something for our\n",
    "# classifier to predict.\n",
    "dataset['DailyResult'] = 0\n",
    "\n",
    "# Now, if we predicted the price to close higher than the open,\n",
    "# put a 1, otherwise put a 0.\n",
    "dataset.loc[dataset['Open'] < dataset['Adj Close'], 'DailyResult'] = 1\n",
    "dataset.loc[dataset['Open'] >= dataset['Adj Close'], 'DailyResult'] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            PrevOpen  PrevHigh  PrevLow  PrevAdjClose        PrevVol  9MA  \\\n",
      "Date                                                                        \n",
      "2002-05-23       nan       nan      nan           nan            nan  nan   \n",
      "2002-05-24      1.16      1.24     1.15          1.20 104,790,000.00  nan   \n",
      "2002-05-28      1.21      1.23     1.20          1.21  11,104,800.00  nan   \n",
      "2002-05-29      1.21      1.23     1.16          1.16   6,609,400.00  nan   \n",
      "2002-05-30      1.16      1.16     1.09          1.10   6,757,800.00  nan   \n",
      "\n",
      "            20MA  UpperBB  MidBB  LowerBB     ...       ATR  BETA  LINEARREG  \\\n",
      "Date                                          ...                              \n",
      "2002-05-23   nan      nan    nan      nan     ...       nan   nan        nan   \n",
      "2002-05-24   nan      nan    nan      nan     ...       nan   nan        nan   \n",
      "2002-05-28   nan      nan    nan      nan     ...       nan   nan        nan   \n",
      "2002-05-29   nan      nan    nan      nan     ...       nan   nan        nan   \n",
      "2002-05-30   nan      nan    nan      nan     ...       nan   nan        nan   \n",
      "\n",
      "            LINEARREG_ANGLE  LINEARREG_SLOPE  STDDEV  TSF  Open  Adj Close  \\\n",
      "Date                                                                         \n",
      "2002-05-23              nan              nan     nan  nan  1.16       1.20   \n",
      "2002-05-24              nan              nan     nan  nan  1.21       1.21   \n",
      "2002-05-28              nan              nan     nan  nan  1.21       1.16   \n",
      "2002-05-29              nan              nan     nan  nan  1.16       1.10   \n",
      "2002-05-30              nan              nan     nan  nan  1.11       1.07   \n",
      "\n",
      "            DailyResult  \n",
      "Date                     \n",
      "2002-05-23            1  \n",
      "2002-05-24            0  \n",
      "2002-05-28            0  \n",
      "2002-05-29            0  \n",
      "2002-05-30            0  \n",
      "\n",
      "[5 rows x 35 columns]\n",
      "Done.\n"
     ]
    }
   ],
   "source": [
    "# Let's preview the new dataset so far. Note the Not a Number (nan)\n",
    "# values. We'll handle these in a second.\n",
    "print(dataset.head(5))\n",
    "\n",
    "# Now save our initial dataset to a .csv file for later review if we want\n",
    "if not os.path.exists(\"./historical data/\"):\n",
    "    os.makedirs(\"./historical data/\")\n",
    "dataset.to_csv('./historical data/' + ticker_input + '_historical_data.csv')\n",
    "\n",
    "print('Done.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. EXPLORE AND PREPROCESS THE DATASET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's change the dataframe's variable to df, an industry standard.\n",
    "df = pd.DataFrame(dataset)\n",
    "df.columns = dataset.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Dataset names : Index(['PrevOpen', 'PrevHigh', 'PrevLow', 'PrevAdjClose', 'PrevVol', '9MA',\n",
      "       '20MA', 'UpperBB', 'MidBB', 'LowerBB', 'SAR', 'ADX', 'APO', 'CCI', 'DX',\n",
      "       'MACD', 'MACDSIGNAL', 'MACDHIST', 'MOM', 'ROC', 'RSI', 'STOCHSLOWK',\n",
      "       'STOCHSLOWD', 'ULTOSC', 'OBV', 'ATR', 'BETA', 'LINEARREG',\n",
      "       'LINEARREG_ANGLE', 'LINEARREG_SLOPE', 'STDDEV', 'TSF', 'Open',\n",
      "       'Adj Close', 'DailyResult'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "# Print the \"feature\" (or column) names.\n",
    "print(\"[INFO] Dataset names : {}\".format(df.columns))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Dataset shape (Rows, Columns) : (4340, 35)\n"
     ]
    }
   ],
   "source": [
    "# Show the \"shape\" of the dataset.\n",
    "print(\"[INFO] Dataset shape (Rows, Columns) : {}\".format(df.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] df type : <class 'pandas.core.frame.DataFrame'>\n"
     ]
    }
   ],
   "source": [
    "# Print the dataframe's type (in case you need to know for later)\n",
    "print(\"[INFO] df type : {}\".format(type(df)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Checking if any columns with NaN values...\n",
      "PrevOpen            True\n",
      "PrevHigh            True\n",
      "PrevLow             True\n",
      "PrevAdjClose        True\n",
      "PrevVol             True\n",
      "9MA                 True\n",
      "20MA                True\n",
      "UpperBB             True\n",
      "MidBB               True\n",
      "LowerBB             True\n",
      "SAR                 True\n",
      "ADX                 True\n",
      "APO                 True\n",
      "CCI                 True\n",
      "DX                  True\n",
      "MACD                True\n",
      "MACDSIGNAL          True\n",
      "MACDHIST            True\n",
      "MOM                 True\n",
      "ROC                 True\n",
      "RSI                 True\n",
      "STOCHSLOWK          True\n",
      "STOCHSLOWD          True\n",
      "ULTOSC              True\n",
      "OBV                 True\n",
      "ATR                 True\n",
      "BETA                True\n",
      "LINEARREG           True\n",
      "LINEARREG_ANGLE     True\n",
      "LINEARREG_SLOPE     True\n",
      "STDDEV              True\n",
      "TSF                 True\n",
      "Open               False\n",
      "Adj Close          False\n",
      "DailyResult        False\n",
      "dtype: bool\n"
     ]
    }
   ],
   "source": [
    "# Now check if any columns have blank values\n",
    "print('[INFO] Checking if any columns with NaN values...')\n",
    "print(pd.isnull(df).any())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice anything? Remember that we shifted the data down one day to put the\n",
    "previous day's data in the current day's row? Well since we don't know the\n",
    "previous day's data from the day before the start date of the dataset, \n",
    "Pandas has no choice but to put 'nan' in there to fill the empty space.\n",
    "We also experience this when creating our 9 and 20 period moving averages\n",
    "and other indicators because we need the previous day's of data before the value for\n",
    "them would start to show up.\n",
    "\n",
    "There are various ways to deal with 'nan' values in a dataset:\n",
    "1. Replace the missing value with a large negative number (e.g. -999), but this is an old way of doing things.\n",
    "2. Replace the missing value with mean of the column.\n",
    "3. Replace the missing value with median of the column.\n",
    "\n",
    "For this application, our nan values are only in the top few rows due to \n",
    "our shifting, so we can just drop these rows from our dataset because 30\n",
    "days max missing from a dataset out of many years of stock data won't make too much\n",
    "difference. If your stock's ETF ticker symbol doesn't date back as far as\n",
    "your stock's historical data, you will have more nan values to deal. The\n",
    "newbie way to do this for now is just dropping every row that has a nan\n",
    "values, so you could employ the commented out code in place of this one,\n",
    "that choice will be up to you, but we don't want any nan values in our data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PrevOpen           False\n",
      "PrevHigh           False\n",
      "PrevLow            False\n",
      "PrevAdjClose       False\n",
      "PrevVol            False\n",
      "9MA                False\n",
      "20MA               False\n",
      "UpperBB            False\n",
      "MidBB              False\n",
      "LowerBB            False\n",
      "SAR                False\n",
      "ADX                False\n",
      "APO                False\n",
      "CCI                False\n",
      "DX                 False\n",
      "MACD               False\n",
      "MACDSIGNAL         False\n",
      "MACDHIST           False\n",
      "MOM                False\n",
      "ROC                False\n",
      "RSI                False\n",
      "STOCHSLOWK         False\n",
      "STOCHSLOWD         False\n",
      "ULTOSC             False\n",
      "OBV                False\n",
      "ATR                False\n",
      "BETA               False\n",
      "LINEARREG          False\n",
      "LINEARREG_ANGLE    False\n",
      "LINEARREG_SLOPE    False\n",
      "STDDEV             False\n",
      "TSF                False\n",
      "Open               False\n",
      "Adj Close          False\n",
      "DailyResult        False\n",
      "dtype: bool\n"
     ]
    }
   ],
   "source": [
    "# Drop all Nan rows of the dataset to get rid of\n",
    "# the ones caused by the shifting of data\n",
    "df = df.dropna(how='any')\n",
    "# df = df.iloc[20:] # to drop the first X amount of rows instead\n",
    "\n",
    "# Now check if we have any nan values. All Falses? Great!\n",
    "print(pd.isnull(df).any())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Dataset types:\n",
      "PrevOpen           float64\n",
      "PrevHigh           float64\n",
      "PrevLow            float64\n",
      "PrevAdjClose       float64\n",
      "PrevVol            float64\n",
      "9MA                float64\n",
      "20MA               float64\n",
      "UpperBB            float64\n",
      "MidBB              float64\n",
      "LowerBB            float64\n",
      "SAR                float64\n",
      "ADX                float64\n",
      "APO                float64\n",
      "CCI                float64\n",
      "DX                 float64\n",
      "MACD               float64\n",
      "MACDSIGNAL         float64\n",
      "MACDHIST           float64\n",
      "MOM                float64\n",
      "ROC                float64\n",
      "RSI                float64\n",
      "STOCHSLOWK         float64\n",
      "STOCHSLOWD         float64\n",
      "ULTOSC             float64\n",
      "OBV                float64\n",
      "ATR                float64\n",
      "BETA               float64\n",
      "LINEARREG          float64\n",
      "LINEARREG_ANGLE    float64\n",
      "LINEARREG_SLOPE    float64\n",
      "STDDEV             float64\n",
      "TSF                float64\n",
      "Open               float64\n",
      "Adj Close          float64\n",
      "DailyResult          int64\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "# Now let's check the dtypes of the dataset\n",
    "print('[INFO] Dataset types:')\n",
    "print(df.dtypes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Print a statistical summary of dataset:\n",
      "       PrevOpen  PrevHigh  PrevLow  PrevAdjClose        PrevVol      9MA  \\\n",
      "count  4,306.00  4,306.00 4,306.00      4,306.00       4,306.00 4,306.00   \n",
      "mean      62.66     63.62    61.62         62.66  18,230,439.85    62.45   \n",
      "std       99.01    100.45    97.37         98.96  20,083,694.03    98.74   \n",
      "min        0.38      0.41     0.35          0.37     285,600.00     0.46   \n",
      "25%        3.69      3.79     3.60          3.70   7,201,075.00     3.70   \n",
      "50%       12.26     12.53    11.98         12.30  12,139,500.00    12.23   \n",
      "75%       81.24     82.41    80.66         81.98  21,770,350.00    80.57   \n",
      "max      421.38    423.21   413.08        418.97 323,414,000.00   406.51   \n",
      "\n",
      "          20MA  UpperBB    MidBB  LowerBB     ...           ATR     BETA  \\\n",
      "count 4,306.00 4,306.00 4,306.00 4,306.00     ...      4,306.00 4,306.00   \n",
      "mean     62.05    67.40    61.98    56.55     ...          2.14     0.65   \n",
      "std      98.27   106.24    98.16    90.36     ...          3.51     0.59   \n",
      "min       0.55     0.69     0.55     0.33     ...          0.05    -2.49   \n",
      "25%       3.68     4.13     3.68     3.17     ...          0.16     0.33   \n",
      "50%      12.16    14.09    12.21     9.62     ...          0.65     0.63   \n",
      "75%      78.95    88.11    78.30    68.99     ...          2.18     0.94   \n",
      "max     403.88   433.48   402.91   381.30     ...         19.91     5.99   \n",
      "\n",
      "       LINEARREG  LINEARREG_ANGLE  LINEARREG_SLOPE   STDDEV      TSF     Open  \\\n",
      "count   4,306.00         4,306.00         4,306.00 4,306.00 4,306.00 4,306.00   \n",
      "mean       62.66             2.59             0.07     1.24    62.73    62.73   \n",
      "std        99.03            23.71             0.88     2.41    99.15    99.08   \n",
      "min         0.38           -81.26            -6.51     0.00     0.35     0.38   \n",
      "25%         3.69            -2.25            -0.04     0.08     3.68     3.69   \n",
      "50%        12.40             0.50             0.01     0.29    12.52    12.28   \n",
      "75%        83.59             6.98             0.12     1.18    83.08    81.62   \n",
      "max       415.54            83.08             8.23    23.65   420.30   421.38   \n",
      "\n",
      "       Adj Close  DailyResult  \n",
      "count   4,306.00     4,306.00  \n",
      "mean       62.73         0.49  \n",
      "std        99.03         0.50  \n",
      "min         0.37         0.00  \n",
      "25%         3.70         0.00  \n",
      "50%        12.32         0.00  \n",
      "75%        82.61         1.00  \n",
      "max       418.97         1.00  \n",
      "\n",
      "[8 rows x 35 columns]\n"
     ]
    }
   ],
   "source": [
    "# All numbers? Good. Machine learning algorithms want numbers\n",
    "# to deal with. If we had categorical (word) values, we'd have\n",
    "# to one-hot encode them. Homework; research what that is :D\n",
    "\n",
    "# Print a statistical summary of the dataset for reference\n",
    "print('[INFO] Print a statistical summary of dataset:')\n",
    "print(df.describe(include='all'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Printing the Pearson Correlation tables...\n",
      "PEARSON CORRELATION\n",
      "                 PrevOpen  PrevHigh  PrevLow  PrevAdjClose  PrevVol   9MA  \\\n",
      "PrevOpen             1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "PrevHigh             1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "PrevLow              1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "PrevAdjClose         1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "PrevVol             -0.16     -0.16    -0.16         -0.16     1.00 -0.16   \n",
      "9MA                  1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "20MA                 1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "UpperBB              1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "MidBB                1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "LowerBB              1.00      1.00     1.00          1.00    -0.17  1.00   \n",
      "SAR                  0.99      0.99     0.99          0.99    -0.16  1.00   \n",
      "ADX                 -0.07     -0.07    -0.07         -0.07     0.16 -0.07   \n",
      "APO                  0.18      0.17     0.18          0.18    -0.05  0.17   \n",
      "CCI                  0.01      0.01     0.01          0.02     0.00 -0.01   \n",
      "DX                  -0.04     -0.04    -0.04         -0.04     0.22 -0.04   \n",
      "MACD                 0.26      0.26     0.27          0.26    -0.07  0.25   \n",
      "MACDSIGNAL           0.29      0.29     0.29          0.29    -0.07  0.28   \n",
      "MACDHIST            -0.03     -0.03    -0.03         -0.03     0.01 -0.06   \n",
      "MOM                  0.11      0.11     0.12          0.12    -0.02  0.08   \n",
      "ROC                 -0.01     -0.01    -0.00         -0.00     0.01 -0.02   \n",
      "RSI                  0.06      0.06     0.07          0.07    -0.03  0.05   \n",
      "STOCHSLOWK           0.07      0.07     0.07          0.07     0.00  0.05   \n",
      "STOCHSLOWD           0.08      0.08     0.08          0.08    -0.01  0.06   \n",
      "ULTOSC               0.09      0.09     0.09          0.09    -0.03  0.07   \n",
      "OBV                  0.69      0.69     0.69          0.69     0.05  0.69   \n",
      "ATR                  0.94      0.94     0.93          0.94    -0.10  0.94   \n",
      "BETA                 0.12      0.12     0.12          0.12     0.04  0.12   \n",
      "LINEARREG            1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "LINEARREG_ANGLE      0.13      0.12     0.13          0.13    -0.04  0.11   \n",
      "LINEARREG_SLOPE      0.13      0.12     0.13          0.13    -0.03  0.10   \n",
      "STDDEV               0.79      0.79     0.79          0.79    -0.01  0.79   \n",
      "TSF                  1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "Open                 1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "Adj Close            1.00      1.00     1.00          1.00    -0.16  1.00   \n",
      "DailyResult         -0.00     -0.00    -0.00         -0.00    -0.03 -0.00   \n",
      "\n",
      "                 20MA  UpperBB  MidBB  LowerBB     ...        ATR  BETA  \\\n",
      "PrevOpen         1.00     1.00   1.00     1.00     ...       0.94  0.12   \n",
      "PrevHigh         1.00     1.00   1.00     1.00     ...       0.94  0.12   \n",
      "PrevLow          1.00     1.00   1.00     1.00     ...       0.93  0.12   \n",
      "PrevAdjClose     1.00     1.00   1.00     1.00     ...       0.94  0.12   \n",
      "PrevVol         -0.16    -0.16  -0.16    -0.17     ...      -0.10  0.04   \n",
      "9MA              1.00     1.00   1.00     1.00     ...       0.94  0.12   \n",
      "20MA             1.00     1.00   1.00     1.00     ...       0.94  0.12   \n",
      "UpperBB          1.00     1.00   1.00     0.99     ...       0.95  0.13   \n",
      "MidBB            1.00     1.00   1.00     1.00     ...       0.94  0.12   \n",
      "LowerBB          1.00     0.99   1.00     1.00     ...       0.93  0.12   \n",
      "SAR              1.00     1.00   1.00     1.00     ...       0.94  0.13   \n",
      "ADX             -0.07    -0.06  -0.07    -0.09     ...      -0.06  0.03   \n",
      "APO              0.13     0.13   0.13     0.12     ...       0.04  0.02   \n",
      "CCI             -0.02    -0.02  -0.02    -0.02     ...      -0.04 -0.18   \n",
      "DX              -0.05    -0.03  -0.05    -0.06     ...      -0.04 -0.06   \n",
      "MACD             0.21     0.21   0.21     0.20     ...       0.09  0.02   \n",
      "MACDSIGNAL       0.26     0.26   0.25     0.25     ...       0.12  0.05   \n",
      "MACDHIST        -0.09    -0.08  -0.09    -0.10     ...      -0.07 -0.09   \n",
      "MOM              0.05     0.06   0.05     0.04     ...       0.01 -0.07   \n",
      "ROC             -0.03    -0.03  -0.03    -0.04     ...      -0.04 -0.13   \n",
      "RSI              0.03     0.03   0.03     0.03     ...      -0.02 -0.11   \n",
      "STOCHSLOWK       0.04     0.04   0.04     0.04     ...       0.03 -0.08   \n",
      "STOCHSLOWD       0.05     0.05   0.05     0.05     ...       0.03 -0.06   \n",
      "ULTOSC           0.06     0.07   0.06     0.06     ...       0.04 -0.07   \n",
      "OBV              0.69     0.69   0.69     0.68     ...       0.64  0.12   \n",
      "ATR              0.94     0.95   0.94     0.93     ...       1.00  0.13   \n",
      "BETA             0.12     0.13   0.12     0.12     ...       0.13  1.00   \n",
      "LINEARREG        1.00     1.00   1.00     0.99     ...       0.94  0.12   \n",
      "LINEARREG_ANGLE  0.07     0.07   0.07     0.07     ...      -0.00 -0.06   \n",
      "LINEARREG_SLOPE  0.07     0.07   0.06     0.05     ...       0.01 -0.04   \n",
      "STDDEV           0.79     0.80   0.79     0.77     ...       0.85  0.14   \n",
      "TSF              1.00     1.00   1.00     0.99     ...       0.94  0.12   \n",
      "Open             1.00     1.00   1.00     0.99     ...       0.94  0.12   \n",
      "Adj Close        1.00     1.00   1.00     0.99     ...       0.94  0.12   \n",
      "DailyResult     -0.00    -0.00  -0.00    -0.00     ...      -0.01  0.01   \n",
      "\n",
      "                 LINEARREG  LINEARREG_ANGLE  LINEARREG_SLOPE  STDDEV   TSF  \\\n",
      "PrevOpen              1.00             0.13             0.13    0.79  1.00   \n",
      "PrevHigh              1.00             0.12             0.12    0.79  1.00   \n",
      "PrevLow               1.00             0.13             0.13    0.79  1.00   \n",
      "PrevAdjClose          1.00             0.13             0.13    0.79  1.00   \n",
      "PrevVol              -0.16            -0.04            -0.03   -0.01 -0.16   \n",
      "9MA                   1.00             0.11             0.10    0.79  1.00   \n",
      "20MA                  1.00             0.07             0.07    0.79  1.00   \n",
      "UpperBB               1.00             0.07             0.07    0.80  1.00   \n",
      "MidBB                 1.00             0.07             0.06    0.79  1.00   \n",
      "LowerBB               0.99             0.07             0.05    0.77  0.99   \n",
      "SAR                   1.00             0.06             0.06    0.79  0.99   \n",
      "ADX                  -0.07             0.08             0.07   -0.02 -0.07   \n",
      "APO                   0.19             0.61             0.66    0.08  0.19   \n",
      "CCI                   0.01             0.46             0.34   -0.01  0.01   \n",
      "DX                   -0.04             0.14             0.11    0.07 -0.04   \n",
      "MACD                  0.27             0.68             0.74    0.12  0.28   \n",
      "MACDSIGNAL            0.30             0.50             0.52    0.14  0.30   \n",
      "MACDHIST             -0.03             0.72             0.82   -0.02 -0.03   \n",
      "MOM                   0.11             0.81             0.91    0.06  0.12   \n",
      "ROC                  -0.01             0.48             0.39    0.00 -0.00   \n",
      "RSI                   0.06             0.54             0.43    0.01  0.07   \n",
      "STOCHSLOWK            0.06             0.31             0.24    0.05  0.06   \n",
      "STOCHSLOWD            0.07             0.41             0.32    0.05  0.07   \n",
      "ULTOSC                0.09             0.45             0.37    0.07  0.09   \n",
      "OBV                   0.69             0.13             0.11    0.54  0.69   \n",
      "ATR                   0.94            -0.00             0.01    0.85  0.94   \n",
      "BETA                  0.12            -0.06            -0.04    0.14  0.12   \n",
      "LINEARREG             1.00             0.13             0.13    0.79  1.00   \n",
      "LINEARREG_ANGLE       0.13             1.00             0.89    0.03  0.14   \n",
      "LINEARREG_SLOPE       0.13             0.89             1.00    0.05  0.14   \n",
      "STDDEV                0.79             0.03             0.05    1.00  0.79   \n",
      "TSF                   1.00             0.14             0.14    0.79  1.00   \n",
      "Open                  1.00             0.13             0.13    0.79  1.00   \n",
      "Adj Close             1.00             0.13             0.13    0.79  1.00   \n",
      "DailyResult          -0.00            -0.02            -0.02   -0.00 -0.00   \n",
      "\n",
      "                 Open  Adj Close  DailyResult  \n",
      "PrevOpen         1.00       1.00        -0.00  \n",
      "PrevHigh         1.00       1.00        -0.00  \n",
      "PrevLow          1.00       1.00        -0.00  \n",
      "PrevAdjClose     1.00       1.00        -0.00  \n",
      "PrevVol         -0.16      -0.16        -0.03  \n",
      "9MA              1.00       1.00        -0.00  \n",
      "20MA             1.00       1.00        -0.00  \n",
      "UpperBB          1.00       1.00        -0.00  \n",
      "MidBB            1.00       1.00        -0.00  \n",
      "LowerBB          0.99       0.99        -0.00  \n",
      "SAR              0.99       0.99        -0.00  \n",
      "ADX             -0.07      -0.07        -0.02  \n",
      "APO              0.18       0.18         0.00  \n",
      "CCI              0.02       0.02        -0.01  \n",
      "DX              -0.04      -0.04        -0.03  \n",
      "MACD             0.26       0.26        -0.01  \n",
      "MACDSIGNAL       0.29       0.29        -0.00  \n",
      "MACDHIST        -0.03      -0.03        -0.02  \n",
      "MOM              0.12       0.12        -0.02  \n",
      "ROC             -0.00      -0.00        -0.02  \n",
      "RSI              0.07       0.07        -0.01  \n",
      "STOCHSLOWK       0.07       0.07        -0.02  \n",
      "STOCHSLOWD       0.08       0.08        -0.01  \n",
      "ULTOSC           0.09       0.09         0.00  \n",
      "OBV              0.69       0.69         0.01  \n",
      "ATR              0.94       0.94        -0.01  \n",
      "BETA             0.12       0.12         0.01  \n",
      "LINEARREG        1.00       1.00        -0.00  \n",
      "LINEARREG_ANGLE  0.13       0.13        -0.02  \n",
      "LINEARREG_SLOPE  0.13       0.13        -0.02  \n",
      "STDDEV           0.79       0.79        -0.00  \n",
      "TSF              1.00       1.00        -0.00  \n",
      "Open             1.00       1.00        -0.00  \n",
      "Adj Close        1.00       1.00         0.01  \n",
      "DailyResult     -0.00       0.01         1.00  \n",
      "\n",
      "[35 rows x 35 columns]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbMAAAFSCAYAAAB49TGgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXmcXEXVv5/vTCZ72Ax7AoGwL2EVWUQiiOKCgLIkoMiiURFRRPyp8CqivqACgmwSERBU9i3yooBA2BEChF0gJCxh3yEh68z5/VHVyU2nb92e6Z7p6eQ8+dxPuutU1a3bt6fPrapT35KZ4TiO4zjNTEujG+A4juM4teLOzHEcx2l63Jk5juM4TY87M8dxHKfpcWfmOI7jND3uzBzHcZymx52Z4ziO02NIOk/S65Iey7FL0h8kTZH0iKQtq6nXnZnjOI7Tk1wA7JawfxZYNx7jgLOrqdSdmeM4jtNjmNntwNuJLHsAF1rgXmA5SasW1btEOzNJ7ZImS3pM0uWSBtap3mUlXSjp2XhcKGnZetTtOI6zlLM68GLm/fSYlqRPtzWndzDLzDYHkPQ34FvAKSWjJAEys45O1vtn4DEzOzDW8wvgXGCfurS6Cua9ObUhOmSDh+2UtM+YflsPtaSMjvaaim+w0X65ti8PXj9Z9p55r+fa3mv/MFn2W20jk/aBiW/mae3TkmVH9V8laX9q7lu5tpX7DE6WbUFJe6vy7a0Fz9DnHTooae+z77hc27d3Oz1Z9pxJv03av731j5L2FKefsHHS3mfXr3W57iJ2HHVI0n7vyxPTN6yAzvze9F1x5DcJw4MlxpvZ+E6crlJbC8+/RPfMyrgDWEfSCElPSjoLeBAYLunTku6R9GDswQ2W9FlJl5UKSxot6R+S1gG2An6Zqft4YGtJI2O+2yVdLekJSX+U1BLrWOw8Mf05Sb+I6Y9K2qDHPhXHcZwiOtqrPsxsvJltnTk648gg9MSGZ94PA14uKrRUODNJfQiTio/GpPUJY7JbADOBY4FPmdmWwCTgB8BNwLaSSo+J+wGXAhsBk81sQVcgvp4MlB7NtgGOAjYFRgJfkjQ05zwl3ozpZwM/rOPlO47j1IZ1VH/UzgTgwBjVuC3wnpm9UlRoSXdmAyRNJjiOFwjDgwDPx4lFgG0JDuqumPdrwJpmNh/4F7B7dIafB64ldIErdXmz6feZ2dTo5C4GPp53nkz5q+L/DwAjKl2MpHGSJkmadO6FF3fiY3Acx6mBjo7qjwIkXQzcA6wvabqkQyV9S9K3YpbrganAFOBPwGHVNHGpmTMrEabJmJlNAm4ys7EVyl8KfIcQeXO/mX0g6XFgC0ktpbm2OIy4GfAkoUtc7uys4DwAc+L/7eTcl9hdHw+NmzNzHGfpw9rn16+u/N/Akt0Iv7udYknvmVXDvcAOcS4MSQMlrRdtE4EtgW8QHBtmNgV4iDBkWOJY4MFoA9hG0lrRye0H3FlwHqeJUeJYrrUuAbRNR0fiaCk4nF5Izw4zdoml/rtjZm8ABwEXS3qE4HQ2iLZ24DrCfNt1mWKHAuvFFerPAuvFtBL3ACcCjwHTgKtT53GWXN4tiGZ0nKagEwEgjWKJHmY0s8Xii83sOWCTsrRbgI/m1HE4cHhZ2jvAVxKn/tDMFov1zjuPmY3IvJ4EjE7U7TiO07M0sMdVLUu0M3Mcx3HqQBWBHY3GnVmdMbOJhLk2x3GcJYLO60r0PO7MHMdxnDR1jGbsLtyZOY7jOGkaGNhRLUutM5PUTlAE6UNYH/Y1M6s59EzSc8DWZvZmrXX1RsISkKWLmkTtCigKJ64l3LiRocrdeu5uHPJSLXe7CeaVukwTDDMuzaH5s8xsczPbBJhLECFeQJRSWZo/H8dxnEAdFUC6C/+xDtRFhDivckkrSLom7pp6r6RRMf1RSctFx/mWpJIK/0WSPtXN1+w4jlMdvmi691NnEeI8fgE8ZGajgJ8CF8b0u4AdCALFU4EdY/q2hEXV5W11bUbHcXqeJuiZLbVzZiwUIYbQM/szsBr5IsQAfYF7zGy+pJII8RUEEeLURkgfB74MYeG0pI/EzTzvAD4BPE9Qyx8naXXgbTObUV6JazM6jtMIrGNeo5tQyNLszOouQpw4V95mc7fHOtYAjgH2AvYmODnHcZzeQRMEtyz1w4wFdEqEOMHtwAGxjtGEvcveN7MXgaHAumY2lSBI/EN6sTOTlDyWRKzgqIWUIG/RsdSilvyjRqyGf7S0pI9mpgnmzJbmnlkhZvaGpIMI4sD9YvKxwNNm1i7pOoJ4cPl+6I9IKt3Vy4DjgPOjwPCHZfn/A7TG13cAJxCcmuM4Tu/A15n1XrpRhHhEzin3yKnjq5nXd+O9ZcdxehtNsM5sqXVmjuM4TpW4nJXjOI7T9DRBAIg7M8dxHCdNEzgzLY1ae5XoDq1GSRcQ1qWdk0nbExhnZp9LlJtRaU4vS7/+w7t841L3vCgiccb025L2wcN26lKboFgXzxKxgx0Ff2wtBdFkh62yQ67txlnTkmWX6zMo1/aFPqsly54784mkPXXNq/dbIVl2ZsecpH1QS79c23sFO2QPau2ftLcn5ljmFKxZGt53+aT9gffz78eoIWsmy05847Gkfd1lV8+1tRZES/ZraUvaH357atJeCxstt0bS/tCrd9UUajzr9guq/r0Z8ImDGhLW7MEGC+kOrcaLgTFlaWNiuuM4TnPQBAog7swqUy+txn8DG0haNaYPBD4FXBPf/0DSY/H4fo9fpeM4TjU0wTozd2Zl1FOr0czagauAfWP6F4FbzewDSVsBBwMfI8hmfUPSFt1+gY7jOJ2lfX71R4NwZ7aQklbjJOAFglYj5Gs1TiYsfl7TzOYDJa3GPgStxmtjmexQY3aI8ePA1WY2M+owXsVCoeGKZIWG29sXk250HMfpHppgmNGjGRfSXVqNdwGrStoM2J6Fjq3Tk6RZoeFaAkAcx3E6hS+aXuK4FzhT0jpmNiXOgQ0zs6cJWo1/pkyr0cwszqf9BbjezGZH0+3ABZJOJDi2vYCvUiVFUYXdRVG0YqPaVSsbbLB3rm2Pwevl2gDunvtqru1vs6ckyx4xcKOkfZmEitAJc/+bLLvZwPzIPIBpc9/Ota3etlyybFHUa2ti0KetII7qokOHJO199j071/ad3U5Pli36fh6y1Q9zbUVPj386cdOkvc+u5xfU0HV2HHVIt9UNNEVovg8zdgIze4OgxXhx1Fm8F9gg2tqB6wjzbdeVFb0Y2Ay4JFPXg8AFwH0EfcZzzeyh7r0Cx3GcLuDDjM1Dd2k1xvSHqDCsaGanAKdU0xbHcZyG0QTDjN4zcxzHcdLUOZpR0m6SnpI0RdKPK9jXkHSrpIckPSIpV2SihDszx3EcJ00dhxkltQJnEqZkNgLGSiqfPD4WuCwuiRoDnFVUrzszx3EcJ019F01vA0wxs6lmNpcQS1C+RZYBy8TXywIvF1Xqc2aO4zhOmvoGdqwOvJh5P50gHpHlOOBGSd8FBhGUk5K4M+sEkoYDFwKrEHauH29mp0lagRCOPwJ4DtjXzN6Ju1SfT1AMuTnWsRdhgfQ+ZnZFTFuR8ORxeFaU2Ok5UoK+H5DeZXe+5dtT9QK8r7S9tSU/BH52e1pIeKalBX3nJdo9N2ED6LNgc/QclP/j11IU4170w9lScO4amJH4zDoKg/OXYDrhzCSNA8ZlksbHNbILslQoVv7hjgUuMLOTJW0HXCRpE7P8rp8PM3aO+cBRZrYhQQ3kO3Gs98fAzWa2LnBzfF/iUcKNKTEGeLis3n0IYf6VFmM7juM0lvb2qg8zG29mW2eO8WW1TQeGZ94PY/FhxEOBywDM7B6gPzA01UR3Zp3AzF6J68OICh9PErrMexAWRRP/3zNT7A5gG0ltkgYD6wCTy6oeCxwFDJOUXunqOI7T09R3ndn9wLqS1pLUl/CAP6EszwvALgCSNiQ4szdSlboz6yKSRgBbEBY8r2xmr0BweMBKmaxGUM//DMHpTSirZziwipndR3gS2S9xzgXajOde6LvIOI7TQ9QxACRq2R4O3EDoEFxmZo9LOl7SF2O2owji6w8TRCcOsoLNN33OrAvEHtaVwPfN7P0iaR9CtM4RhKico4CfZmxjiN3pmO/PVFhIDYtqM857c+pSPIDvOE6PUmdlDzO7Hri+LO1nmddPAPk75lbAnVknkdRGcGR/M7OrYvJrklY1s1fi3mWvZ8uY2X2SNiGIGT9d5vzGAitLOiC+X03Sumb2TDdfiuM4TnWkO0W9AndmnUDBC/0ZeDJKUZWYQNgO5sT4/7UViv8EmJ1NkLQ+MMjMVs+k/YLQW/tlfVtfH9R5sf+eoSMdfVcUAddH+fZaxuJT9dZad9GIQEsN96qK0Ybuo6XgU2ltzM9W4efZ1tYzDWkETSA07M6sc+xAULZ/NO5nBmHI8ETgMkmHEiYu9ykvaGb/rFDfWODqsrQrCcONvdKZOY6zFNLATTerxZ1ZJzCzO8nfh2yXCvkvICjjl6cfFF9eUcH2CEHixXEcp1dgHT7M6DiO4zQ7PszoOI7jND1NsAWMOzPHcRwnjQ8zNj+SDPirmX01vu8DvAL8x8y+EBf5bWRmJ1YoO8PMBscF1k8CTxHm3GYCB5vZU5JGE6IfpxGC214H9jez18vr6w0UaQ02jBr1+lL6irU8k6bqrabu1KddsIa0Ji3BorprCWot/DyLhrQaFIxQ+HnOS2thNjXze38AiCuAFDMT2ETSgPh+V+ClktHMJlRyZBV41sw2N7PNCJJX2YXTd0TbKILUy3fq1HbHcZzaMav+aBDuzKrjn8Dn4+uxBHkVACQdJOmM+HotSfdIul9SKrR+GeCd8sS4jm1IJZvjOE7DqK82Y7fgzqw6LgHGSOoPjCLoMVbiNOBsM/so8GqZbaSkyZKeBX7AopJVO8Z1ay8Q9u05r66tdxzHqYUOq/5oEO7MqiCu/RpB6JVdn8i6Awt7bReV2UrDjCOB7xM1FiOlYcbhhP3PflupchcadhynIdR3p+luwQNAqmcCcBIwGvhIIl81jyYTCE4rz3ZlxYpdaNhxnEbg0YxLFOcB75nZozECsRJ3EXQV/wockJMH4OPAs12wLaRIi7Cb6OjOMfEa9RVrqXt+R360VnsNUYHzCs5by09Ee8FTcHvBZHx3RqYmoyFrlX2cl95hO0nB/UhFLBZ+D+YWtKtBf7P1wOb3/ra7M6sSM5tOmBNL8T3g75K+x+K9q5FxXkzAXODrGduOGdt7ZTbHcZzG4oummx8zG1whbSIwMb6+gKi/aGbTgO0yWU+M6c8BA6hArGvZerXXcRyn7vgwo+M4jtP0uDaj4ziO0/R4z8xxHMdpenzObOlD0jHA/kA7QYbum2b2H0krAi8Dh5vZOZn8zwEfEALb3gEONLPne7zhVdJStAtwTZV3X7RiUd1tLfm7BA8mXXZeQn+xb0v6T6ytILSvb+KBuCiasShasaMG6aEizclW5X9PWmuNomzrl2sq3Am9hu9YYd1989vV3XT3zuDNEM3oi6briKTtgC8AW0adxU8BL0bzPsC9hIXX5Xwy5p8IHNsDTXUcx6keVwBZ6lgVeNPM5gCY2Ztm9nK0jQWOAoZJWj2n/D1Ans1xHKcxuDNb6rgRGC7paUlnSdoJQNJwYBUzuw+4DNgvp/xuwDU901THcZwqaQI5K3dmdcTMZgBbAeOAN4BLJR1EUAW5LGa7hMWHGm+V9DphWPLvefUvqs14Sb2b7ziOU5km6Jl5AEidMbN2wtzXREmPAl8jDB2uLKkkcbWapHXN7Jn4/pOEfdMuAI4nqOpXqnuhNuPrz/T+WFnHcZYIbH7vj2b0nlkdkbS+pHUzSZsTHhgGmdnqZjbCzEYAJxB6awsws1kENf0DJa3QU212HMcppAn2M1Ph9uhO1UjaCjgdWA6YD0whhOPPMLMfZ/KNAi4xs41iaP7WZvZmtJ0OvG5mqc09GTl0y4bcuC8OXi9pnzDj6VxbUah4H6XDplPh4CmhYEiH3gP8979X5Nq+vvXRybKzE+3606c/TJb93k1DkvZUOPi0+e8lyw5q6Zu0L6N8+4YMTJa9rePNpL2/8gd9+hfc55kd85L2aXPyzz2sX/o5cNqsN5L2zQYNS9pTvDjv3aT93Xkzc221LJMAWL5tUNI+6ZU7aord/+Cwz1bdwCFn/bN71wnk4MOMdcTMHgC2ryLfI8BG8fWIMtt3u6VxjuM4XaUJFEB8mNFxHMdJYmZVH9UgaTdJT0maIunHOXn2lfSEpMcl5QbGlfCemeM4jpOmjj0zSa3AmcCuwHTgfkkTzOyJTJ51gZ8AO5jZO5JWKqrXnZnjOI6TpM7RjNsAU8xsKoCkS4A9gCcyeb4BnGlm7wCY2etFlfowo+M4jpOmvuvMVmehzB+E3lm58tF6wHqS7pJ0r6Tdiir1nlmdkbQXcBWwoZn9V9II4EngKaAvcDtwmJl1SNqYEP04jLDL9IXAr6yKgecvD16/621M2IpO/H+zpibteySiHT8gLVZa9GSVejYs2tK+SCw4FbF47qTfJcset3W+nOZBN6afaP9+7cFJu72b/0B6y343JctuvGo64vCid5bLte06b1ay7NMF0XNDEtGMIywdZflCazqasSOh5zvH0mXHDN4gaf/RJ/M/79ah6cjTgy9bPmlfv29+pOTcGsWX75j3ak3lC+lEx0zSOIJwRInxcY3sgiwVipV/AH2AdYHRhN/HOyRtYma5IaPeM6s/Y4E7WXQd2bNmtjkwihDFuKekAcAE4EQzWw/YjBAJeVgPt9dxHCeJdVj1h9l4M9s6c4wvq246MDzzfhhhCVN5nmvNbJ6ZTSN0BtYlgTuzOiJpMLADcChli6IBzGw+cDewDmGbmLvM7MZo+xA4HKgY2eM4jtMw6jvMeD+wrqS1JPUl/FZOKMtzDUEZCUlDCcOOyWEhd2b1ZU/gX2b2NPC2pC2zRkkDgV2AR4GNgQeydjN7FhgsaZlKlWe1GR/+YEq3XIDjOM5idHTiKCA+1B8O3ECYgrnMzB6XdLykL8ZsNwBvSXoCuBU42szeStXrc2b1ZSxwanxdEhQ+ExgpaTJhXPhaM/unpE+TP0VVMT2rzfijEWN7/ypGx3GWCGx+fX9uzOx64PqytJ9lXhtBo7aiTm0l3JnVCUkfAXYGNpFkQCvBKZ3FwjmzLI8DnyirY22C9NUHPdBkx3GcqrAmUABxbcY6IembhB2mv5lJu42wc/TZZrZJWf4BBIc2zsz+Hd9fDtxgZqcXnW/H1XfJvXHdKYxWFDXYntjPKKWt2N3MKzj3Bv3y12Suo7RO4XGTfpVrO2vLn+XaAFoL/vz+m4js2292+poGtaUj+zY4Mv+atXqBRuGH+TqDAB2vvpZf94D+ybL7nfJS0j51Tr6+4lPvTE+W/diK6Sjg84bkh0o+/W5+9CfA8S3pdvdXvj5orX+zsy2tTXrfy7fVdIq399ipakexwrW1naur+JxZ/RgLXF2WdiXw00qZo0r+HsCxkp4izKPdD5zRnY10HMfpLE2wN6cPM9YLMxtdIe0PwB8SZR4lrKNwHMfpvfT+7czcmTmO4zhpCkYxewXuzBzHcZwkjRw+rBZ3Zo7jOE4Sd2YOAJLaCQEebYQdqP8CnBr1Gb8EfMfMdol5P04IAtk6Li6syHvt6R2Mu4ux/UYm7X+bnb+Yuzt3mp7XkY7s69uS/qqndoQu0ldMRSwe9uDxybL7b3Vk0j47cV3n9EtrHLYl9BEBhv8h/5rXnvdMsuwzbV3/dWsrGLOaU2BvVX7c2jrLrZYs+0F7WnPytNlDc22z+s9Nlh3WsWzS/uzctFZmLbR2cyxfMzgzj2bsGWaZ2eZmtjFhD5/PAT8HMLOrgNmS9pfUh7Au7bCUI3Mcx+lRTNUfDcJ7Zj2Mmb0eVaXvl3RcXOn+XeDfBImr+83s7oY20nEcJ0Mz9MzcmTUAM5sqqQVYCXgtvr+UoFeWHsdzHMfpYTrmN67HVS0+zNg4Fnw7omP7FDADWDO3QEZo+O0P8xUWHMdx6omZqj4ahTuzBhA1GNuB0k6A3wEeI2wdc6akit+I7D5BKwxcuWca6zjOUo8rgDiLIWlF4I/AGWZmklYhKENvY2ZvSPoG8HXgT6l6vtXW9dHI1BNM0Xfx5JlPJO1HDNwo1/a+0tGMtew0XSQc11agfve9m/Kj3Ip2gz5390tybeds8TNua8nXjf77A79P1t3xXv7Ox7dvny677T5pveq/XTsg17bdkORuGzwzO72r8vId+XdzlYLQpta2jyTtT/bLb/fL7elrHssqSftXDsxvnFbMj3QEOOLk/HsF8Lk++X+z82rs0FzQ/mJtFRRgHb1/mNGdWc8wIG4BUwrNvwg4JdpOAX5rZiX11O8Ttgi/0sze7vmmOvUk5cgcp1loBj16d2Y9gJnlLqAys/3L3r8IjOjuNjmO41SL98wcx3Gcpqej3Z2Z4ziO0+R4z8xxHMdpehoZcl8t7swcx3GcJK4A0kUkGfBXM/tqfN8HeAX4j5l9IZPvWmAlM9uurPyBwI8IC5MFnGdmJ0m6ANgJeB8YANwL/MTMXorlDgGOJER6twDHmNm1sdx1ZnZFbMvxwD5Aae/4y83s15m2n2JmR8X3PwQGm9lxmfY9DDxhZmMzaQvOUc1nNLCGL1ctoflFYsHLJPR+W1tqe7qrJaCqb0Fh9clvm72bDrn+b+u8XFtKKBjSofcALcuulGs7o9/MXBvAkKv6J+1P980PQ9/gncHJslP7p4WuB7e05dre75sWSB5c0AsYmBCkHtiSrntQwbKAt65/I9fWd+CrybJt5N8rSP/N1hqa31Kw9KRWOpqgZ9ZbF03PBDaRVFpQsivwUjaDpOWALYHlJK2VSf8sIbz901HYd0vgvUzRo81sM2B94CHgVkl9JQ0DjgE+bmajgG2BRyq07VfAasCmZrY5sCMh5L7EHOBLkiouSpG0IeFz/4SkQcUfheM4TmPpaG+p+mgUvdWZAfwT+Hx8PRa4uMz+ZeAfwCXAmEz6T4AfmtnLAGY228wWW4Bsgd8DrwKfJegkfkCQlMLMZpjZtGwZSQOBbwDfNbPZMd8H2V4XYR3ZeEIPrxL7E9aZ3Qh8Me/iHcdxegtm1R+Nojc7s0uAMZL6A6OA/5TZSw7u4vi6xCbAA504z4PABsDDwGvANEnnS9q9Qt51gBfMrGgl7JnAAZIqbXC0H3BphXYXktVmnDgzvd+U4zhOvbAOVX00il7rzMzsEcLi4bHA9VmbpJUJjuVOM3samC9pky6eSvF87cBuwN7A08DvJR2XLCgdLGmypBclDc+0/X3gQuCIsvwfBd4ws+eBm4EtJaV1gTJktRlHD1q32mKO4zg10WGq+mgUvdaZRSYAJ7H4EON+wPKEXtRzBKdXGmp8HNiqE+fYAngSFgw93mdmJ8T6vlyWdwqwhqQhMf/5cd7sPaB8VvpUgnBwdl5sLLBBbPOzwDIVzuE4jtOraAbV/F4ZzZjhPOA9M3tU0uhM+lhgNzO7ByAGgNwEHAucAPxW0hfM7FVJ/YBvmtkfshVHZfrvAqsC/5K0GrCKmT0Ys2wOPJ8tY2YfSvozcIakb5rZbEmtwGIhVGb2tqTLCA7tvLjNyz7AqEz05Cdjm8/t7AdzWvu04kzdwOr9VkjaT5j731zb7PY5ybI5mwUswBID8u0FscNF9g0HD8u13bLfTcmy+yk/YvGcfunouiKx4FTE4hUP/iHXBjD/0VuS9rljb8i1zVjs2WxRxszJj1YEWJ78CM8Om5ss+6PWdITnm3Pfz7XN7cg/L8BLfd9N2r80Jl8oWyPXSZa9/7D/S9rv7cb49lZ1b7/EtRlrxMymA6dl0ySNANYghNWX8k2T9L6kj5nZ9XEY8t/RYRnBKZb4naT/AQbGOj5pZnMltQEnRac2G3gD+FaFZh0D/BJ4TNIHwCzgL8DLFfKeTNhwE+ATwEslRxa5HdhI0qrx/TmSTo2vXyxfcuA4jtMI2hO7IPQWeqUzM7PFFrmY2URgYny7egX7lpnX5wPnV8hzUOKczwM759gOyryeB/w4Hsm2m9lrBKdZYtuyvO2EniFAbtscx3EaSb17ZpJ2I3RUWoFzzezEnHx7A5cDHzWzSak6e6UzcxzHcXoP9QzsiFMzZxLWD08H7pc0wcyeKMs3hBBEVx7JXpHe33d0HMdxGkqdA0C2AaaY2VQzm0tYhrVHhXy/BH5LmPYpxJ2Z4ziOk6TOofmrA9mtsadTNnUkaQtguJldV20bfZixBqrRkJS0J0HLsS8wD/gfM7sm2i4A9gVWLi3ElnQaoWu9opm9mXfuUf3T27+nqOUJ5uE5ryXtmw1cbDpzATMtHWlWpC/XkVBnbC8Y1C/SlOyb0PvbeNXc2wDA228NzLW1Kf0ntu0+6fX3KX3FomjFPptWnAJewPIDrs21rdgnrSn59AfLJe3zEz9qg1rTAonrtX0kae+X+ExfmpPenH3z/qsm7amIRa2ZH+kIsGrbnUn7ii3597Lo+1lE0d9lrXSmdZLGAeMySePNbHw2S+oUMfL793QyjsCdWW0s0JA0s1mUaUhK2oywTm7XGHG5FnCTpKlxUTiEtWt7AH+NN/GTlOlQOo7jNJLORDNGxzU+kWU6MDzzfhiLRoMPISg5TYzLdVYBJkj6YioIxIcZayelIflD4H9LGo/x/xOAozN5LiYsAgcYDdxF0Hd0HMfpFXR04qiC+4F1Ja0lqS9BoGJCyWhm75nZUDMbYWYjCEuoko4M3JnVg5SG5MYsrhM5KaaXeAZYMcpajY31OY7j9BoMVX0U1mU2n7D+9gaC+tJlZva4pOMldVl83YcZa8TMHokLuRfTkCSMDZcPN1dKu4rwdPIx4Jt558qORW+7whasN2StvKyO4zh1o6PO68zM7HrKfi/N7Gc5eUdXU6f3zOpDnobk48DWZWlbAk+UpV1CCEO9ySxf8yYrNOyOzHGcnqIDVX00Cu+Z1Yc8DcmTgMsl3WJmz8Ue3E8JyvwLMLMXJB0D/LvaEz41962aG90VBrX0S9qnzc2PJptn6Qi5WiiKBusoiHYclYhyu+iddOTeUUfmRzMO/0N6R+a/XTsgaU/tBp3SVoR0tCJfDV1nAAAgAElEQVTABveflmub/3+p+XtY+Zp7kvY+Q/O/J60j0pG4756T/m6/Mz9fr/LdOTOSZZ/qm6771eNuzrW98vr9ybJvt6Tv9Rvz022rhe7WZmxvoJOqFndmdaCShmRMnyzp/wH/iNqP84AfmdnkCnnP6f6WOo7jdJ5q5sIajTuzGqhCQxIzu4owJ1ap/EE56SPq0T7HcZx60H16//XDnZnjOI6TxJ2Z4ziO0/T4MKPjOI7T9HT0fl/mzqxZWbnPYtN1PcK0gijK1dvSkX9zExGNtew0XSsbkh+RuOu8WcmyWn29XNtPfgOX/uCZXPt2Q9Kf5wbv5N/nDuDDxI7QRfqKRRGLfT4/LtfW9tDjybJqyb+X9ta7tG6+ca5905a0AM6rLfk7Ta89eBWe/zB/p+oVC/5uBq+Ur5W57kpv8tLTy+baW+env79Du/Fv9rVujJSE5ohm9HVm3YykdkmTJT0m6R+SlovpLZL+ENMflXR/1G5E0nOShja25fUn5ciWVFKOrFZSjqxWUo6sVlKOrFZSjqxWUo5sSafOclbdgjuz7meWmW1uZpsAbwPfien7AasBo8xsU2Av4N0GtdFxHCeXDqnqo1H4MGPPcg9BvxFgVeCVkuJHXKvmOI7T6+i+Af764T2zHiJuFb4LC9WhLwN2j0OQJ8fN6IrqGCdpkqRJz814oTub6ziOswAfZnQABkiaDLwFrADcBAt6YusDPyF8B26WtEuqoqw244jBa3Rzsx3HcQLzpaqPRuHOrPuZZWabA2sSdpsuzZlhZnPM7J9mdjTwv8CeDWqj4zhOLtaJo1H4nFkPYWbvSToCuFbS2cCmwKtm9nLcYXoU8EiykgwtDQqVHdSav/U7pMPr+9QafVfDJc8viKS8rePNXNvTbYOSZcd/mC98+0xbeuDlmdnLJ+1T++eL146Z05Ys+/QH6WUSKbHgotD7/sfmixQDtE8t38ZvIR3335Ys+zJzkvZlW/PFmYe05S+xqIZbHhuWa3u5Lf0FHNyaXlKQ+ib0qfHvea51736+vs7MWQQze0jSw4S9y94A/iSpJC9+H3BGwxrnOI6Tg8tZOYuJEZvZ7pm3/8opM6I72+Q4jtMZmiGa0Z2Z4ziOk8SHGR3HcZymp3tn5OqDOzPHcRwniXnPrDokHQPsD7QT5hrfAZYHBgMrAtNi1sOAScBvgd1j3ieA75QUNCStApwKfBSYAzwHfB+YC1wXZaVK5z0OmGFmJ0nalrBbdL94XGpmx0k6CNjazA4va/OywOnADjHpLuC7MWrxauAvZnZNzPsUcJGZ/Sq+vxL4G0He6lpgKjAQeA34rZldV/SZtSaiBosma2tZj9Fu6dpbU7Wr+6aRi0SIi7aV76v8SMshSv+ZdLz6WtKeYvmOdLsGt+RHLC7PvGTZ+QW/QH2G9su1pYSCIR2tCNC69la5NnsyXbathsi+Pon7CMVRgyOULyrdOi8/ihLA+qS/g20t+fe61jVSRd/vWvEAkCqQtB3wBWBLM5sTBXb7xpD10cAPzewLmfwnAUOA9cysXdLBwFWSPhazlBzJmJh/c2Bl4MWCpvwF2NfMHo5qHesX5P8z8JiZHRjP8wvgXGAf4G5ge+AaSR8BZgDbZcpuR1hvtgFwR+n6YluvkTTLzG4uOL/jOE6P0AzOrDcsml4VeNPM5gCY2Ztm9nKljJIGAgcDR5qFhUNmdj6hB7Yz8Elgnpn9sVTGzCab2R1VtGMl4JVYpt3MnsjLKGkdYCvgl5nk44GtJY0k9NK2j+nbA9cBKyqwFmEh9avl9ZrZ5FjP4eU2x3GcRtEMi6Z7gzO7ERgu6WlJZ0naKZF3HeAFMyvf0GgSsDGwCZAawxgZtRAnR4mpb2VsvweeknS1pG9KSq0O3giYXHKoEBwgMDm24wFgE0l9Cc7sHuApYMP4/q5E3Q8SemyO4zi9gg5VfzSKhjszM5tB6OWMIywkvjTOU1VCVHb+eenlPBu3Y9k8Skxle3DHA1sTnOv+5KwBq6YdsZf5OLAlsC3wH4JD2z4edxfUXdmQERqeOuP5RBWO4zj1Y34njkbRcGcGC4b1JprZzwlDbF/OyToFWFPSkLL0LQmBII8THGNX2/GsmZ1NULffLM53VeJxYIsoQwWEzTaBzYAnY9LdwCeAIWb2DnAvC51Zqme2RaaO8vYtEBpee/Canbgyx3GcrtMMw4y9IQBkfaDDzEpb8m4OVOx2mNlMSX8BTpH0rRgAciAhEvCWmO1/JX3DzP4U6/9otCe7MpI+D1xvISxuXUJkZcXNMs1siqSHgGMJc1zE1w+a2ZT4/i7gZGBifP8IoZe2MsEZVmrDKOB/gK+n2grpqMG0Yl9tzOlIR9C1JaKqWmr8picnoQuGN1oL/sz6J6LgRljfZFkNyB+RbivQzFul4FH2/b755+6wucmygwq0AltHrJJraxm+erJskb5iKmKxaBfrlX9+bNL+YuJeFUXbpqJWAZYZNDvXtvz76b+sAYnIU4B+3dh3KIrirBVfNF0dg4HTJS1H6KVOIQw55vET4CTgaUkdwH+BvaITQtJewKmSfgzMZmFofhFfBX4v6cPYjgOiswQ4SFJW0X5b4NDY7imEn9J7YlqJu4G1gRMAzGy+pNeBF0sbckZ2jI5xIPA6cIRHMjqO05tohmjGhjszM3uAhZF/5baJLOzZlNLmAN+NR6UyLwP75pxuk7K8x2Vej8mp7wLggpz6vpKTjpm9Tll/wcxGl72fCCybV4fjOE5voN7Dh5J2I6zrbQXONbMTy+w/IIxQzSfEUhxiZsnRtV4xZ+Y4juP0XuZjVR9FxHW8ZwKfJUSGj5W0UVm2hwhiFaOAKwhCGUncmTmO4zhJ6hwAsg0wxcymmtlc4BJgj0XOZ3armZU287sXyN9oLuLOzHEcx0nS0YmjClZnUUWm6TEtj0OBfxZV2vA5syUJScMI3eeNCA8K1wFHE+YEryVoTLYQAj32JwR93AmskQ0KiQu6x5nZfXnnOu/Q9O7H3cV+56Z3Rr7o0PJVExk6Cr7qCe26qsrXwN5/rhi4CsALrekIzv1OeSnXNqcgmrG1LW/1R2BwQl/xR62vJ8uuV1D3u+e8lWvbtCXd7qLdoFP6ikXRisdN+lXSPvv4I3Jt35uwdrLs6Yfk61EC/OC8/IjED/unvwfnrPde0j5wx8LORZf59DnlOhL1pTPRjJLGsWgQ33gzG5/NUqFYxU6dpK8Q1v+mxDQA75nVDYWwx6uAa8xsXWA9QqTmr2OWO+Ji7VHA/QRx5OcITyg7ZurZgLA2LdeROY7j9CQdWNVHdj1sPMaXVTcdGJ55PwxYTMJQ0qeAY4AvluQOU7gzqx87A7OjVmRJ3upI4BBCDwxY4PSGEHYGALgYyEZSjolpjuM4vYI6z5ndD6wraa0o+TcGmJDNIGkL4ByCI0sPQUR8mLF+lDQZF2Bm70t6gaApuWMcPvwIMBP4acx2GfCQpO+a2XxgP4LyvuM4Tq+gmijFaolrbg8HbiCE5p9nZo9LOh6YZGYTgN8RRrYuj2t9XzCzL6bqdWdWP4p0I7Nbvfw/Qqjpt8zsVUmPA7tIeo2g+v9YxRNkxqL/sPs2HLL1Ot1wGY7jOItS73VmZnY9cH1Z2s8yrz/V2Tp9mLF+PE6YqFyApGUIY8PPluWdQNBtLFEaakwOMWbHot2ROY7TU9Q5mrFbcGdWP24GBkatyNLCwJMJ6iEfluX9OIs6uCuBzxGGGC/p9pY6juN0gs4EgDQKH2asE2ZmURfyLEn/Q3hQuJ4wN7YdC+fMBLxHRkzYzN6VdC+wsplNq+Z8ffZNi7WmG5t4firYfv2BU76dtPfZ9+x8Y0uBGGprwdexPREuPq8g2KktHZI97awjc20d6aJMn/t2rq1oO/sn+w1I2gcmBGTfnJsOx+6n9Of5zvyZubZXW9J1L9uabneKlFAwpEPvAfr/7A+5trYJ/y9Ztujv5uE//jzXViRSPPjo/ZL2luEb5hsLvidFtCfaXQ8aqYZfLe7M6oiZvQjsXsE0kQINRjPbI2V3HMdpFC407DiO4zQ97U3QN3Nn5jiO4yRp5FxYtbgzcxzHcZL0flfmzsxxHMcpoBl6ZoobNDudRFI78CghOrEdONzM7pY0AngSeCqT/RTgO0A/YAVgAFBSp93TzJ6L8i0PAruZ2Q1F5z9kxN4NuXEvtedHwAEMb22MAHIRSgjfAkxrz4/em2/tybJ3vv5krm2d5VZLll2lb3pv1oEtfXNtj8x4IVm2b0u+aC7Au3Nm5NpaC0Sfh7QNTNr7JCL/2lPRtMBOg9NiwSkR4zMm/SZZ9ptb/yhpv2NmfjBx35b0s//HB6yRtM/rRofw1Lz8iFqAu166pRNSwYvzjRH7VN34Pz13eU3n6ireM+s6s8xscwBJnwFOYKGy87MlW4YLY96DCJvOHV5mH0tQ0B9LkHlxHMfpFXgAyNLDMiwUDu40UXx4b2BX4A5J/c1sdr0a5ziOUwvWBM7MFUC6zgBJkyX9FzgX+GXGNjLaSseOOXWU2AGYZmbPEtakfa5SJknjJE2SNOmpD6bW4xocx3EKaQY5K++ZdZ3sMON2wIWSNom2SsOMKcayUMbqEuCrhL3RFiHuCzQeGjdn5jjO0kdHE8RWuDOrA2Z2j6ShwIqdLRs1HL8MfFHSMYSAko9IGmJmH9S5qY7jOJ2m97syd2Z1Ie4O3Qq8RWYjzir5FPCwmX0mU99fgD2Bi/IKnTPpt11oae0MHpbevXzG9Nt6qCVldKQjDot0ITfYYO9c25jBGyTLzlsxf3Dlg/ZZybJjWSVpH5SQo3yp77vJspv3XzVpf6rvW7m2FfsMTpYtok8i4rBI4/D0Q9JimCl9xaJoxaK/m8O3Tms7pjj11+nvSZ/PHNzluovYcdQh3VY3NEdovjuzrjMgCgdD6E19zcza40ZyIzM2CJvP5amjjgWuLku7Evg2CWfmOI7TU3g04xKMmVV8vDSz5wjryPLKXUDYFqb0/qAKeSZQto244zhOo/CemeM4jtP0NENovjszx3EcJ4lvAeM4juM0Pc0ge+jajE3K12tYZ5bSKSwaTrh31vSkfYv++VqEM2xeumE1UOuYfmpH6PN3/jBZ9tV78jUQT5s9JFn2dwemdQveuv6NXNvQMWsly2rkOkn7q8fdnGsbvNLcZNlbHhuWtI9QfhTnMoPS4janzkvrez48N/8zeWteejXLzoPSuo8pbcf2aQ8ly352rz8m7Rv2WT7XNrfGvs+Dc15N21+5sya9xD3W+ELVf2DXvnBdQ7QZCxVAJC2mRirpOEk/jK8vkPSSpH7x/VBJz8XXIyTNKlPDODBTzxaSLGobZutvj3kfk/QPSctVqO8JSRdKaou20ZLeKzvXp6JtZUl/lzRV0gOS7pG0VxXXflq8tpZM2kGSOiSNyqQ9FgWGkTRY0tmSnpX0UDzfNzLtf6zCeS6QNC3T7ruL2uY4jtNTtGNVH42iXnJW7UDeQodnzWzzzHFhxpYV180yK+bdBHiboDi/SH3ApsAwYN+M7Y6yc/076h5eA9xuZmub2VbAmFg2l+jA9gJeBD5RZp4OHJNT9FyCTuO6ZrYFsBtBKb+IozPt3r6K/I7jOD1CB1b10Sjq5cxOBY6UVPUcXEZc9yDg05L652S9B1i9PNHM2oH7KtnK2BmYa2YLxgDM7HkzO72g3CeBx4CzWdzZXgdsLGn9bKKkkcA2wLFmYZ8LM3vDzNL7UjiO4/RizKzqo1HUy5m9QOhhfbWCLU90t1BcN0o97UKFNVfR+X0M+Fcmeceyc40ENibsE9ZZxgIXExY0f6E0nBnpAH4L/LSszMYENY+uDID/LtPuv1XKkBUa/q8LDTuO00M0g9BwPVXz/xc4ukKd5cOMd8T0cnHdbO+npK7xFmGI7qaMbWTG9oKZPZKxlQ8zPlveSElnSnpY0v15FyKpL8G5XmNm7wP/AT5dlu3vwLaScmfhJR0TndPLeXkyZIcZD6iUwczGm9nWZrb1BkPSE9mO4zj1wjrxr1HULTTfzKZEJ7NvUd4qxHVnmdnmkpYlDOl9ByjJQT0bbasCEyV9MSpm5PF4PFepnd+JosCTEmV2A5YFHo3yVAOBD4H/y9QzX9LJQFbM7QlgM0ktZtZhZr8Gfl0piKZZSUX9QVqQtGg8vaVgN+hU+aKJ56KdplNqga1D0xGJT7+bN0IOs/qnowK14tCkve/A/Ci1omhFrblR0v7K67nPcwx8Nx15+nJbwec5L1cEh+XfT++A/WH/9LlT2o5Fu0EXkYpYbF1ri2TZAQU7ezczRbuD9wbqvZ/Zr4EfVpGvJK473MxGmNmaBD3CPbOZzOw94Ajgh2XDfJjZK8CPgZ8UnOsWoL+kb2fSisSAxwJfj20bAaxFmNcrL3dBvJYVY5umEJzkr6LDLg2HNiRU1XEcpx4sKQEgAyVNzxw/yMtoZo+z+PxU+ZzZEeSL6+5foc6HgIcJEYjlXBPbV5qHK58z29vCjOSewE4x/P0+4C8s2qNaQHRYn2HRXthMwpzg7mVtm0voMa6USf468BFgiqQHgH+XnWv9ss9zn5j+u7K2963UPsdxnJ6m3sOMknaT9JSkKZJ+XMHeT9Kl0f6f0tKnFIV9cjNLOrxyoVwz+1Lm9XMkRHfLyi0Q1zWzwWW2rBPZJJNuwGYZ27I5db9CZWdYKe+HVAilz14XiwoF/4GFQ6DEObZv5tT9HFBpLOLyatrmOI7TCOq5OWcctToT2JWwzOl+SRPM7IlMtkOBd8xsHUljgN8A+6Xqrfcwo+M4jrOEYZ04qmAbYIqZTY2jW5cAe5Tl2YMwggZwBbBLXM6Vy1KtzaigPFK+BmyamRWqgziO4ywt1HkubHWCGEWJ6YRlVhXzxGC79wjTN2/mVbpUOzMzuwG4odHtcBzH6c10JppR0jggux34eDMbn81SoVi5t6wmzyIs1c4MwlowQuBJO2HN3zvA8sBgQpTitJj1MMJaulWBOUBfQnDHsWb2bqyrHXiUMC82n9BNPtXMOiSNBq7N1Ach8vNY4IToWEtt+j6wnpkdltfu00/YuOsX3ZH4YrakR553Oipf5BXgTydu2pUWBdoKQpvnJUK2585Jl+3bL2n+xFG35toOvixfIBbguZaXcm3DOipO4y7giJNfT9rbFoktWpT7D/u/XBvAqm13Ju1vt+QLKLfOTwfgDm6dn7Rbn/zfnaIQ9nPWey997qPzp06+Py7/PgKc+usNkvaUWHBRu6998Iykff4N5+cbC/7uivjkUbkdlrrQmZ5ZdFzjE1mmA8Mz74cB5WtxS3mmR2WpZQnShrks1c5M0nbAF4AtzWxOXH/W18xejs7nh2b2hUx+gAPMbFKMNjyB4KB2illmRd1IJK1EWFi9LPDzaL8jW1/MN5IQnJLtIY4hLEB3HMdpOHVeDH0/sG4UnHiJ8HtXHsk+AfgaQc5wb+AWK9DKWtoDQFYF3jSzOQBm9qaZVaPWUQrL/xGwhqTNKthfJ3S1Dy+YuLyCIJdV2nVgBLAaYSmA4zhOw6mnNqOZzQcOJzzAPwlcZmaPSzpe0hdjtj8ThDSmAD8grClOsrQ7sxuB4ZKelnSWpJ0KS2SIYscPAxXHLsxsKuEzLo0VLaYdaWZvEQSTd4t5xgCXVnoKyWoz/vnmBzrTVMdxnC5T70XTZna9ma1nZiOjUhJm9rOSmpOZzTazfcxsHTPbJv6WJlmqnZmZzQC2IvSg3gAulXRQJ6spUvfI2vO0Iy9m4Tq4MfF9pfYu0GY8dJetOtlMx3GcrtFuHVUfjWKpdmYQeldmNtHMfk7o+n65qEyJuPhvU0JXuZJ9bUJgSXqWPyiZ7CJpS2CAmXVF5d9xHKdbWKqEhpuRuB9Zh5k9E5M2B56vsmwbQYvyxTLl/pJ9ReCPwBlmZqlpMzObIWkicB45vbJy+uz6tWqy1Z2H3z4vae+zayJiqzvpaK+p+Lvzrsu1rd83uY8rryZ0pJ+dm44y+1yfkUn7wMSD7r0FT8ErtuQLIAO8MT+/3UP7DM61QfFWH22J6Lx+Bc/QA3dMf94twzfMtc3jlmTZPp85OGnf8JinkvYUyWjFKs5dG7d1Y931VQDpLpZqZ0YIvz9d0nKEUPopLLo+ohJ/kzQH6EcIzc+uXC9tXVMKzb8IOCVj3zHaS/zKzK6Iry8GrqJK2S3HcZyeopE9rmpZqp2ZmT0AbJ9jm0jYNDSbNrqgvty9KWJ9uYuOzOxqXF3fcZxeiPfMHMdxnKbHe2aO4zhO09MMm3O6M3Mcx3GS+DDjEoykYYQ9eTYiLHG4Djg6KoM4TUbqj3VuwRBLLROd8woKF9lTdOfQUJ+Cq+7WNT/qvtrnFsZpJqhRX7E30wzDjEvup9+NRHmqq4BrzGxdYD1CZOSvG9owx3GcbsCso+qjUXjPrGvsDMw2s/MhLLyWdCQwTdI04DOE0P21gL+b2S8AJH0FOIKguP8f4LBYdgZwGkH0eBawh5m91tMX5TiOU4k672fWLXjPrGtsDCwijmhm7wMvEB4QtgEOICzC3kfS1pI2JGz7vUNU1m+PeQAGAfea2WbA7cA3euQqHMdxqqCeQsPdhTuzriEqbxRXSr/JzN4ys1mE4ciPA7sQdCDvjwundwHWjuXmEubcIDjJERVPmhEaPvfCqoRCHMdxaqYZtBl9mLFrPE6ZhqOkZQibybWzuKMzgqP7i5n9pEJ98zIq+e3k3Jfspnfz3pza+/v9juMsETRDNKP3zLrGzcBASQfCAsHhk4ELgA+BXSWtIGkAsCdwVyyzd9y0k2hfsxGNdxxnyaJVLcmjVppBaNidWReIvai9CPNhzwBPA7OBn8YsdxJ0GScDV5rZJDN7AjgWuFHSI8BNhM1BHcdxejXNMGfmw4xdxMxeBHYvT4/q+K+b2eEVylwKXFohfXDm9RWE3acdx3F6Bc0QzejOzHEcx0nS3uFyVksdZnYBYe7McRxniaCRw4fV4s7McRzHSdIMw4xqBo/rLM62q41uyI2b0zEvae/X0tZDLekcqZ2+AeZ1zM+1tbWkn/mK1tbUUndLgQbifMvfYbvoBygV5dZaEBs21/KvqajuPsrd9g8ovubU510UuVd0r4quK0VfdV/foOi6Jj58btLeNnTtmvZKXGbQ2lX/3rw/c2pD9mVs6mhGSe2SJkt6XNLDkn4gpe+6pNUkXRFfj5Z0XUH+4yS9FM/zhKSx9byGeI7nJA2VtJykw+pdv9M4Uo6sVlKOzHHqSYdZ1UejaGpnBswys83NbGNgV+BzwM9TBczsZTPbu5Pn+X2UoNoDOEdSd3U/lgPcmTmO06vwdWY9iJm9DowDDldghKQ7JD0Yj+0BYvpj2bKSWiQ9I2nFzPspkoaWneMZwqLo5WO+kZL+JemBeK4NYvo+kh6LvcXbY9pBks7InPM6SaPLLuNEYGTsBf6ujh+P4zhOl2nv6Kj6aBRLVACImU2Nw4wrAa8Du5rZbEnrAhcDW+eU65D0V4Lw76nAp4CHzezN7FyLpC2BZ6LjhCAt9S0ze0bSx4CzCIr6PwM+Y2YvSVquE5fwY2CT2AtcDEnjCA6btZZdl5UGrtaJqh3HcbpGM+xntkQ5s0jJ+7QBZ0gqKdSvV1DuPOBagjM7BDg/YztS0jcIwsC7AUgaDGwPXJ5xeP3i/3cBF0i6jCA0XBey2oyNCgBxHGfpoxkCBZcoZyZpbYLjep0wd/YasBlhOHV2qqyZvSjpNUk7Ax9j4fYsEObMTpL0JeBCSSNjne9W6kWZ2bdiT+3zwOToUOez6LBu/65ep+M4Tk/SDM6sU5pbve0AZmRerwjcCPwivv89cFR8fTALJBUZATwWX48GrsvU8WXgZeA3mbTjgB9m3l8LfDO+vhvYJ74WsFl8PTKT/yHCvmYfj/lbCOr67wOjY57ngKHAR4Dnu/hZjKvhc2xI2WZt99J4zd7upafdzXo0ewDIgFJoPvBvojOLtrOAr0m6lzDEOLOK+iYAg1l0iLGc44HSEoADgEMlPUzYFmaPmOd3kh6NgSa3Aw8Thh6nAY8CJwEPlldsZm8Bd8Xgkc4GgIzrZP7eULaR5/Zrbp5ze7t7/txNR1MPM5pZ7upLC5GHozJJP4npzwGbxNcTgYmZPJsRAj/+m6nnuLJ6HwDWj2+nEefQyvJ8KadZB1RKNLMRmdf755R1HMdxcmhqZ1ZPJP0Y+DY5DsdxHMfpvTT7MGPdMLMTzWxNM7uz0W3pIuObsGwjz+3X3Dzn9nb3/LmbDtdmdBzHcZoe75k5juM4TY87M8dxHKfpcWfmOI7jND3uzJZSJO0saWCj29FMSBosaVAny+T+jXVSt3OpQFJF/dRGI3XjZmVOXXBn1sRIWk/SnyTdKOmW0lFl8YMIUlv3SPqtpN0lLd+NzV2MuOvAseW7GOTkHSrp55KOiE7l7Li4/FpJ63Tx/MtJOqaKfIdJegF4HnhR0vOd2HduUpQ2K6/z61RYOF8tku4qsK+QOqqov9byn5R0lcJeg49LuqLCLhGV+FPcweJ4SRtVkb8QSYMkfUXS/xXkuzPz+qIy830FZc8o7czRVSStKOmnksZLOq90dKL8b6pJW1Lxp43m5nLgj8CfCJqUVWNmB0LYrBTYGzgTWI3Ed0LSB7CIfLbie4UqbZmi80paFdgP2J+wqP0EoJoNT/8OTALWJfywnA+cBuwInEuQJss753DgfwjXd02s65fAVwm7KaTaeyxBUHq0mU2NaWsDp0lawcx+VdDuI4Dxku4D/h+wJkGdZjrwiYKyKdYosL8Zz1HaHTS7+68RRLNTPMDCe1tOsrykzwNnENRyfhHr2BI4T9LhZnZ9Xlkz20LS+sAY4ApJcwn36BIze76gzdk29CXsb7g/QdjgSsLfSopsr3vj8ioLyj4DnBy/35cCF5vZ5GrbG7kWuIOgZtSVnVd3JXzHsny2QtqSSaP1tPzo+gE8UEPZrwDnEPQiJwA/AuXruc0AACAASURBVLbrxrZ+A7gFeBr4FcGRTetE+Yfj/wJeKLNNLih7K0Fj8zMEzc5HCD+Qq1Rx3qeA/hXSBwBPV9n21njN7wAvAp+uw+f5QoH9NIKM2lkEh68e/F5OJOqUlqWPAm7rZF2bER54ngXuqiL/roQdMF4C/grsDjxX5bkerPS60vtEHWsSnMdDwJOE7aDWq7Js8nucKPdtgkzezPjdLh3TgL/21H1v9OE9s+bmH3G462pgTinRzN6uouyphB+IPwK3WpD5qhpJmxF+JAFuN7NHCoqcCdwD7G9mk2IdnVnk2A6h+yfpzTJb0Y6AK9hCWbIbJL0GfNTM5iTKLMDMFttxwcxmSap2J8J9CL3Pswl75e0naVLRfYq7NFQ0EZxpLmb2PYW9iUYTeqCnS7oRONvMphU1WNKahF0h3ovvPwnsSRDFPtPM5iaKr2JmD1do0yOSVi46d6YNpb0JVyb0mt6ootgNhN7Nx0vXKem0Kk+5nKS9CNMvy2U+fwHLVlOBhd7jb4DfSNqC4Fh/TnigKeI6SZ+zRM81h78D/yQ4/R9n0j+o8rdgicCdWXPztfj/0Zm0aoaQMLOhkjYmDHX9WmED06fM7KtFZSV9j9DTKu3V9jdJ483s9ESx1Qg/6qfEH7TLCHvOVcvakiYQflhKr4nv16qizcuzcKjoVWBgKZij4A9+uqRdzOzmsvp2Bl6p4rz/BmYBnzKzaXGO7nDgfkm/sbBHXR67J2zXFZ3bwmP7rZIeIgzb/ZIwHPanorKE+7MX8J7CFkaXE34sNyf09r6eKJsS9S4U/Ja0I8H57wk8BlwCHFlyrAVsRbjWf0uaGstW40gAbgO+mHmd/fxvr6YCSW2EYc0xwC6xnl8kCy3ke8BP49DqXKofvm8l7MLxnQrtWWFpcWiuALKUImkZYAdgJ0IPayhwr5l9LVkwlH2EMCQ5M74fBNxjZqPSJReUH0b4Yx8LDASuNrOfFpTZKWU3s9sSZZ8j9N4qzv+YWWr+Z2PCXMadLJxH+ijhs9vDzB4vaPdeZnZ1hfRVgJPNrEtaoJJWNrPXEvZBhF0c9iNsj3QVcKmZvVhl/Y+U7qekk4AOM/tR7C1NTt1rSe9S+cdfhB5TbqCRpBeBFwhO6LLUNVZxDTsQvmNfBiYTvmddknmS9GUzuzJh3zWe6/OEOd1LgGtKfyPdiaRpLJzLLv+OJ7/fSxLuzJoYhdD6HwBrmNm42Lta38wKn9qjQ7ozHreb2fROnPdRwjDd7Pi+P3C/mW3ahWtYDxhrZtU+vfY48fr2JwQFiLDdz98qDT8W1LMigJlVM1xWqfyyhB/m/YENzWz1RN6ZhF7YxcAUFg3cwcySO6BLerR0PyU9CPzEzG6I7x8pcGa1PHisaZ0I9KhQfg0ze6EsrYUwlzbGzA7uYr0vmFlu0I2kWwnDfVd2tScUh4UPANYys1/GwKVVzSwZSekEfJixuTmf0FsohQRPJwwHVTMEVXrqHkLZD12V5/2PpFKPY0/gz6kCifkfCJPXSaKj/ikhiOIUwlDZjoR5v0NL83DVorBb+BiCI92kIPtuwArAjaUf9E6e6zjC0KKAFknzgdPN7Pgqyg4gDH3tT4gIHEL4vIuGvS4n3NcN4pHFWDhEnMctki4jDKUuTwjeKUWjpubLiPnvNrPXC/JVYqaknxPu83nA71h4n48ysykF5a8hfE4LMLMOwlxap+9dhqJoxl+VhqIlrZWdl5T0paKHh8hZhBGEnQlDwjMIc80fraqBUsXoWDOraoi02fGeWRMTgwi2lvSQmW0R0x42s82qKLsJcBHhR1qEyfWvmVnhmq9YfivCUJsIPbuHCvJ3EIZ6SuHKi4SKm9khBeXvBC4ElgGOBL4P/IPwQ/crM1tsLVeFOiotC7jKzHKdqaSzCD2yuwlzIP8ws18WnStT/khCiPi4TEDC2oRgkH+Z2e8TZf9GmNO8kTBsdQswxcwK5whrJfYS9gNWJQz3vRTTtwBWSjl1SVcA2wEfEjalvZsQiZgcko1lbyQswRhC+LzPZ+F9PsDMRheUX/C3UE+q6Jk9aGZblr+u9L6ojq78Pce8/8i87Q9sQ4h43rma8s2O98yam7nxyT3ErIfeRlUReoQtIn5gZrfGsqNjWu7CT4VFp38nzAU8QOgVVsuXCT+OowhzUBdX8ZSdZXBpvkPSt8zs8ph+kwp25Zb0DcJ8xjBCYMPXgWurHNr8BCHMvD0O695BeGqulgOBXc1sQQSmmU2V9BWCk8p1ZoRNZN8hhHj/N7ah6qfP+MByNMEZG/AEcFLKeWfaaAQHWs4jhB5tquze8fwjCN+n7YFvSlqDMBz9uUTxlc3sp9GZPm9mpXv7/9s783DLyurM/94qhpSAhEZEQ6MoiKCIKGhQjGkURBQFRRkcCA1x6CQIIgrSIQJGQxwQBaUlMVBtJyARGRQhQAQUBIICyiigoOKIiHEoItPbf6zvcHcdztl7n3ur7j67av2e5z61h/Odverec/ba3/rWetctkh6V4DCCDSR9osa2d4w7V8Lno36/IjIq69CY7VH743hA0kJmvs/r0Zyp+wi2l0oYKmHKD7Ud33fSmfWb9wHnAxuWp/jtCGWPNqwxcGQQXbfVLNV0EnEjO06hNHIq8OWGNO3B+58JnFlJTPiopHWB/123hlKh+qX+dc25UcylLOB+24OygCXlJjsJq1Yd2QDbd5fMt7HYfrakzYiZ5EWSfg6sJekJtn9aN1bSrsBHiNnnR4kb6tbAFyQdYvvshvGPJbLjNiDqEC8kQqWHELPrf64bX+y/s6w3Lio/g+065lKCAZE5OslDVpVdZjkOlnaCw5+ttp+1TxBlNutL+gAhZvDXc7DpLuKBaKUgw4w9pziEbYmb1ZWjbpxjxp1JyCkNZHveBGxje7cWYwfrOHsR4aQvEzOtC1uMXchM6vIWwGFt1qEkLSESGQRsXLYp+0+1PdYRS3ocM7Veg7KAfW1vOMF1B9caXFtEhl9tCKguxNQ2/FR5/TaEY3sdcJftuln0t4hsyzuHjm9EzEqb7D6bmBVeQYT71gFWAw50g7KFpMOJz8V6RNH5leXn24MHg5qxg0xIEaHFwXpPYyZkGT/R73RZMVe7K++zGfH7BviK7ZsnsOF4ZhznAqKM4k7bb2r7Hn0mnVnPKYkVLyI+xJd5RBr4mHHrEPUvLyqHvgocaftXE15/S2AxsKXtsfU8iqLbvYk4/kWEPFHrpA1FEe9Y2mbAacKygDHXFRGyPLwhZIakhxhdWyVCWWSSWrvBewp4cUNW4E22R2ob1p2rvKaazbiQkMd6ku3ftLDvFiJ54UvEetlVblcjVs2EXERIlz1MJH/cB/WZkGX8lba3bXOtEWP3JwrsP1z2f0Ss3Ql4j+0TW9g9kpbRByQ9l5nv8+W2W+t3SqqW1TxIOLJaDc8ViQwz9piSnLAJM/qCb5O0g+3GtQXb9xK6gdX3+wgRRmq67vrAHoRTeCKROdeU8vzvxHrLZcDqwD6S9qnYM3Yto5yfS7r284Af2v6p7bsk3U3IHa0CrNn2uori4TcQ//c7CL2/WuocfAu7/6bhJXU3yAc0Ok39yczoNdbxwGCjrNXd0caRlddvphAjfiGhQHKYpDUJea2v2z65ZvjXgQ8A+xH1ZoMHh1OIbNYmThhsSNquejNX6EKeMHoYAG8nogYDfm57gxIqvYBI2hlJ1VlpliUY5e/9euJzJeBkSf/qZv3PgQ2LK++1DtAYeViRyJlZj5F0I7BFWawf1NNcb3tYJLXt+zVlbA0SKTYjvnCntX3yG3pqfBTVL+KY8cMix8Pjx6okKOqkdrD9S0X68mnAAUQYZvNBwsKYsZsyM5O7hxCRPcR27UxxWSDpXSMOrwHsD6xre6wjlrQbsfj/QZYu9j4MONT2WQ3Xrs4oB/JZS8q2637fQ++zCrFW92LgbUQNVd0M/mPEA8bBA+dZ1u8+Atxn+8CG6806q1DSN21vXdk/3PYHy/bVtsemyJfZ8t8QnysRYb7WJRjlPW4GnuOZ+s1FhCbk5i3HX0KE/1ch1jXvJrQwD24zvu/kzKzffIdQTx/MHjYkZj+zpSm54YXAMcBFjtqd1jQ5qxbj1wKQdDQhR/VZwt43EqGgOhZ6ppB1T+Akh5rDGZKalM1vITIYX+WSfalIt1/u2P7oYFtRD3ggMQM+jUjqqBt7lkIZ4l3M3GBvBPbwCN3EEePnMqN8NfFZ2Y7IpLyRmHG9q/xbxy6EMO8jDy62fy3pfxF/i1pnxtyyCpfSX6w4sgXAug1jDyLCg8/zUAmGpHe6pgSjwp1EksygGH91IsTalrXL7+rPgZNtv08hjrBSkM6s36wL3KxoLwLx5H2Fim6h7VcPD9D4XlSi4cvuop6g6CF2GnCOW8r1KNQr3ksU/K5XDv+cSNM/ZoK1up28dE3ZiZKuoj4FeaGkVWw/SCyuv7Vyruk7sDsxM7tY0vmEI5k0o3HWlL/XwYTTXgw8t4SIGylOa5/GFy579iWc1nuIOqfGbNcKrjqyysG2ZQlzySq8QNLf2h7OIDyaCDPWMZcSjAG/B26UNEik2gG4TKXUoCkUD6yiqKXcA2js07eikc6s3zStqYyirk9V25vOscQM55jiSD8HfMn18k6nE0W//8MlrVyhT7gvsea2Y8trPyTpjYRTMRH+a+r9dCpwaUn1vo+YaaFo6lmbmOClSwp2Iwq215d0IpE80nSTmzWK+rnXEiURz7L92wnGnlN3ftSDzrLC9muLDU8BXlac0M0u/eAauEnSPrb/b/VgcQq3tBi/WZmNCNi4MjMRzQLc7wb+UdLtxPoeRAuab1AvrAxzKMGo8G/E2vLDxGf64vqXP4qjy3tcbvvqMjO8bcL36C25ZtZjys1iUBDb9maxLK+/kJDeeQvw8oZ1q+/Yfvqk50a8diOiV9d2lIwv4CA3tLCRtC2RrHKBZwSSNyWKsSfq+FxmS68H9vRyVFdQqKb8nlh7eVRT1Ibf991E77RTgasYenhpm103G0pI9DPEWtm3yrWfTTxI7W97uE6wOnYDQmprUC82WOtbBLzGRYmkZvycs16LExisO99kuzHUV7ce12KtbhVibXM/YslgAbFkcDKRMfvAuLHJDOnMekhZEP9HZnGzqLzH2cTs5mzbS2ZhwyKiRcaehBbel2wfUPP6C4iU/MUuSuglK3JfIjyzw6Q2JOMpDxoDJfctgXOJWsBGSallcO1TiPWfowdrqyVB4ghgE5cu5w3v8RIqws4easEzoT2PA+4ZFb4cel1t9+7hzNChscMJMzATAaktwShJL2sRbW6Gk16W2D6ozq7K+2xKZFyub3sLRdnMq9tmQ/addGY9ZBndLP6UcESDlhVtQoWDsZ8D/phQHzkduKQpIaSkCh9GqH+sT3zRf0aoS/y9mxtVvsf2h7R0YegjtFhP6D1l1jJIzPhxWQNsM251wql9mPjM1PWdmzOSbrP9tEnPLaNrb0skKf2SkB37LNHeaAGwj+3za8YO5KyW0g0l1ngfP5ekmAabb2Mo6aUcX0jImLX6fUm6lAiVftoz2o43uFlIe4Ug18z6yXa2960eKF+Eo8sXo5ESZrp0KFT4T4SQbxMnE9JQTWtV1evdK+lkQhbpyur6j6SXE46xjoESwkTq+H1G0nuJtZhBavcVxBrfqkQyyN81jF+deFjZG9iIkEtqo94+V+YtQWYEJxD1aGsTa7Q7275SoaxxKjWfMw+1MCoh7UOJRIwPtrm4pGcx06XgppYz4bkmvQx4jO3/0NKKa60eeFYIbOdPz34I5fRx526b4H0WEZlPZxBFwMe3HPcYQjPupLL/NGCXhjHvIEoJziJmlbtWzl3T9e90Gn8IubE1KvvXln8XEmovdWMXE2HnvyVqEefT7sVEcpKGjh8BfHY5X/u6yvbNQ+eubfkeTyOKtG8mEj9WbTFmbeAS4HuEvuJZRFr9xcBjG8aeRcwah4+/icgYbvt/P4+QW7um7L8OOG8+//Zd/uTMrJ9crlALeL/LpxZA0hGEBl4jQ6HCT9IiVFhhNn3U3gJsbfu35Yn385I2sv1xWjzJd5md1yVeuvTh4+XYQ2XNso43E2s4mwLvqDytT1T0PEsOIBJAbi91fAaeA1xLc1bgXKl+hu8bOte0ZrYFkdL+TKLUY3+3jz68n4gavMQzof8FRMjzA8TvZBx/SQhA78eIpJeW1x+8z0lERuePiAfUlUKXEXLNrJeUxeHPEIkXwzeL/d1CB6+E9i6c4MtaHTtxHzUN6QEq5I0+T7QleYntrRqu2Vl2XldIuhV4poey2Ur48AYvx7WnZYGiJdEzmEni+K6kP7L94+V4zUEiRlW1BNolYjxEfMbOZUS5h+vbx9xE6JM+OHR8FUKVp1HFY1klvZQykgVuKT+2opAzsx7iyFZ8/dDN4lC3SCGu8FXgvQr9vrcqOjk/3XZjl2pm10ftp5K2clFcLzO0XYh1umfVDwXgCcxk572BeczO65DPA59WaAougUduVCeUc1NN+TwOfyavJFRrltc155KkUdsgtoH7hx1ZsedBSa16DNr+CqWj96SUte91bP/C9u8kraaQnzu4jSNdEUhn1m+OJbIQz3ZLJY4KswkVDphNH7V9GFqMLl/+fSR9uumCZQZ5PnB+JTvvEknLPTuvQ44gQlQ/kPR94qFlQ2JWfkSXhs2BLpNDavEYyTWF0PCrRp2r8AeKLtyjJLRWXwbmjUXSXsCngd+VBLAjiSzOqwnlmJWCDDP2mDmm108cKiyvGaiYL2EWfdTmwojsvHOAf3JDIW3fKbPgTcru7baH14J6gxrErKeFMtN5GfFZ2wn4musFqWvVOmxvv2wtXOraNwC72b5d0ULmCmAvt2wHtaKQM7Me47ml188mVIhtSzrLoS5+7qyNnxBJi4lmnucBR9m+Yb6u3RWKXnXDPG2QzGF7PtLsJ2ZcLSDx4POH82zORCi6KryBmQfE7Qil/yZhgffabpV8tRy430UE2/Y1inY9K5Ujg3RmvWeEEkdbdfrZhAoHXCnpebavntDcudBldl5X1IW2zPzUjM2GulrAqa0TlHQX0UPtRODdtn9THEMbhZxPEd+/Lni8pGqblzWr+7aP7cCmeSfDjD1mNkocZdycQoUlc+vpRL3YIHPMtrecxX8jWQkoCjC/8hTfcCR9nBCTvh74F6Kjw/W2mwSKqYbr5xtJ76s7b/uo+bKlS9KZ9Zg5ptcv1YhwwrEjxVw9h27QyaOR9Crg24Pfa6kt3J0Qoz3QpW/WtFHsPN32LWWd8zyiEeqDhHLMRZ0aWEN50NueWCt7BRGy3x/4smu6Fkj6FZEhPJL5qIOU9N/cIAu3IpPOrMdIegzR62ri9HpJnwROmSRUWLK63k4kI1wPfGZUOnKybFC0L9nW9pJSxnAscZN9DvB62zt1auAYVOmALumthM07ECHixbaf36mBLVG0bnk5Yf/LbD+u5rW3UVMQPh91kMWG64hM5fOmeRa8PMg1s34zl/T67YG3S7qT9qHCxcADRD+wnYkat6bOv8nscWW95rXEw8M3gW9K+osO7Wri/sqNdCfgtBI9uLkUEfeCUqz+ReCLVcUVSWfY3n3o5b+dgsL9TYmHhv2A48syxCm2b+3WrPmhNx+sZCQb295T0t4Atu/TkMpoDTvP4nrPcBFjlfQZItsrWX6oKKUsITpkf6py7g+6MakVvy/SUD8jHpoOqZx7TDcmzY2hcohRa2j3SnqCZxrP7sNMSPjI+Qj/lQeIC4ELJW0P/D/gLyR9CzjM9hXL24YuSWfWbyZOr59jqPARWaWibDAro5PWHEeEjX5NiOZ+A6AU5/6kS8MaOIhQKFkP+NhgbU/SKwjJtb4zKnz3h5RO7SW9/xhCj3ErQi9xbI3askLSuoQW45uJB4kDiFrMrYiIzVOWtw1dkmtmPUbSjoR6/TOACyjp9bYvqRnzOZYOFX7fdqtQoR7dgHCgfbcip8d3iqKH2eOBb3lGwPaJhJL72GaRyfJDIzpHS7rORV+0rEffbfvI4XPL2a5bCeWPk23fNXTuUNt/v7xt6JJ0Zj1ltun1kq6vhApXAf5j+IuZTA/lb7QzMz2ybgbOn+bEm6Gap0fR97qnUWn4RYVjqxKxuAV4q+2vDs55HhpkStLKlvRRJcOMPWUOShwZKuwJkv6I6If1EyI8J2AX4KOStvdyVJ+fI2tVtt9G6AauSBw64tiphBrPL4jWM18DkLQJ0VB1uSHpi8wsNTzq/HyUBUwDOTPrMbNMr89QYU+QdArRbPK4oePvIHrD/Vknhk1Al8XEkyJpV+C/2/5k2b+KWPcDeI/t2k4FkrYFnghc4CL8LWlTYE3b1yxHu/+07vwUZFnOC+nMekwqcazYSLrF9mZjzn3H9tPn26ZJGbW+NK1IupwQ6P1h2b+OyCJdg1iHemmX9iX1ZJix38wmvT7pD3Xq+G30ApPJWG3gyAqX2b4HuEfRR24qkXS67T0kXc+ITMuV5eE2nVkPSSWOlYa1xyjni3adETph6Ka6SVEygemPHKxT3bH9V5Xd9ZheBtnIu3RqRcekM+snqcSxcnAp45Xzx+oATgF9valeJekttv+helDS25higQDbPyn/rtTaqLlm1kMyvT6ZZkoG3/q2Lx86/ifAj21/txvL6pH0eOAsQnhgkLCxNdEpejfbP+vKtjaUBJTjgc2B1YCFwO9WlqSuBV0bkMyKpdLruzQkWX5IOljS/iOOHyDpoC5saslxwG9GHL+vnJtKbP/c9guB9xNJVXcCR9t+wbQ7ssIJhCjybUSW8p8Tzm2lIGdmPSTT61cOSiHuc23fP3R8deDqaV17qisSrkYVpg1JL7H9lbL9FFda7Eh6rae0s/cASd+wvY2kbw8+G5K+Xhz0Ck+umfUQ2wu7tiGZFzzsyMrB308gKN0FdSLIi2rOdc1HmOkWfQZLd47+a6a3s/eAJZJWA66T9CGi2H5qszCXNRlmTJIpRtL6bY5NGVdLesvwwRIy/WYH9rRFY7ZH7U8jbybu6X9FRG42JJT7VwpyZpYk08uHgXMlvYulExI+RMwippWDgDMlvZEZ57UNkZTwms6sasZjtkftTx22vy9pvbJ9VNf2zDe5ZpYkU4yknYHDgC2IG+qNwDG2z+vUsBaUnlqDtbMbB+tR04qkXxElDwL+hJnyBwEvsr3OuLFdUkLO7yNmZCJmZw8Cx9s+ukvb5pN0ZknSQyQ9bxJNzi4ozuyZhBO+yfbFHZtUS181DiW9E3gFodQ/6B33VOBEosPCx7q0b75IZ5YkPUHSM4C9iPTr/7S9TccmjaT0YPsC8F9EmFFEMsUi4DW2f9ShebNC0nbDdXPTgqRrgR2H2z+VkOMFfRF6niu5ZpYkU4ykJxPOa28idPRkYBvbd3ZpVwMnACfaPqV6UNI+wKeAXbswqglJC4E9gA2IGc0NknYBDicc8bQ6hVVH9TG0fbekVbswqAtyZpYkU4qkrwNrA6cBp9m+TdIdtp/SsWm11Cn6T7Paf2m5syEhXfXHwPeBFwCH2T6rQ9NqqetM0KeuBXMlZ2ZJMr3cTXQTX58Qur2NHmTVETJKj0LSgnHnpoRtgC1tP1zEvH8BbGL7px3b1cSzJf16xHFRX/O3QpF1ZkkypdjeFXgWkZZ/lKQ7gHUkPb9byxr5kqR/qLZNKdv/B/hyd2Y1cr/thwFs/xdwaw8cGbYX2n7siJ+1bGeYMUmS6aIUS+9JJIFsaHvDjk0aSVmn+TtgXyJUZ2KtbzFw+ChVk2lA0hLg9sEusHHZn/bWNQnpzJKkl0h68rS2/JC0re0rJS0ieu4JuN32VDcULck2Y5nW33cSpDNLkilF0jl1522/er5smYQVLemgZDnuZfufu7YlGU8mgCTJ9PIC4IfAqcBV9EMfsLdIeizwl0Rq/jnAhYSqxiHAdUA6sykmZ2ZJMqWUGcGORI3ZlsC5wKm2b+zUsAYqslAjmeIZ5dnAvcAVwEuBdQg9yQNtX9elbUkz6cySpAeUHmZ7E+LDR9ue2qaLkm4jGkOOZIploaod3BcSqflPsj2q0WgyZWSYMUmmmOLEXkk4so2ATzD9fbV+O60Oq4FqB/eHSoF6OrKekM4sSaYUSYsJ1fnzgKNs39CxSW25V9ITBjVaRcZqdyJN/0jbv+zUuvFUi48FLCr72cG9B2SYMUmmFEkPE00WYWnlj6m+uUq6BtjB9i8lvZiQ4zoA2ArY3PbrOjUwWSHJmVmSTCm2+6rQs6Ay+9oTOMn2GcAZknqVSFGUS3YD3mD7lV3bk4ynr1+WJEmml1UkDR6UXwpUm3JO/QO0pNUk7SbpdOAnwA6EFFcyxUz9BytJkt5xKnCppF8A9wFfA5C0CfCfXRpWh6RBGcROwMXAZ4Hn2/6fnRqWtCLXzJIkWeZI2hZ4ItEc8nfl2KbAmrav6dS4MZQ1yq8B+1Y6Nn/P9lO7tSxpQ87MkiRZ5ti+csSxW7uwZQK2JkScL5L0PSJxZZpb1iQVcmaWJEkyhKTtiJDj7oSU1Zm2T+rWqqSOdGZJkiRjKA1FdwT2tL1f1/Yk48lsxiRJEkDSmyrb2wHYftj2vxENUpMpJmdmSZIkLN26ZriNzYrW1mZFJGdmSZIkgcZsj9pPpox0ZkmSJIHHbI/aT6aMDDMmSZIAkpYAtxOzsI3LNmX/qbbX6Mq2pJmsM0uSJAk279qAZPbkzCxJkiTpPTkzS5IkASTdwYhWO2Xbtjeef6uStqQzS5IkCbYZ2l8A7AEcAlw7/+Ykk5DOLEmSBLB9Dzyi+vFm4N2ElNUrbd/UpW1JM+nMkiRJAEmrAvsB7wQuA3a1/d1urUrakgkgSZIkgKS7gAeB44AfDJ+3/YV5NyppTTqzJEkSQNIpjC+OdgoNTzfpzJIkSZLek2tmSZIkgKSD687bPna+bEkmJ51ZkiRJsFbNuQxhTTkZZkySJGlA0kG2j+vajmQ86cySxmey7AAAAGlJREFUJEkakPQD20/q2o5kPNkCJkmSpJnsZzblpDNLkiRpJkNYU04mgCRJkgCSfsNopyVg0Tybk0xIrpklSZIkvSfDjEmSJEnvSWeWJEmS9J50ZkmSJEnvSWeWJEmS9J50ZkmSJEnv+f8oX1h+qTIpDgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1b2cd6c4470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1b2c808f668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# It's good to know what these values are. You can see if there's\n",
    "# a huge outlier from human error that needs to be dealt with\n",
    "# in your dataset, etc.\n",
    "\n",
    "# Finding correlation between attributes and plotting heatmaps,\n",
    "# then saving them in our main directory for later if wanted.\n",
    "print('[INFO] Printing the Pearson Correlation tables...')\n",
    "print(\"PEARSON CORRELATION\")\n",
    "print(df.corr(method=\"pearson\"))\n",
    "sns.heatmap(df.corr(method=\"pearson\"))\n",
    "\n",
    "# Create the plots directory\n",
    "if not os.path.exists(\"./plots/\"):\n",
    "    os.makedirs(\"./plots/\")\n",
    "\n",
    "plt.savefig(\"./plots/heatmap_pearson.png\")\n",
    "plt.show() # comment this out if using .py file\n",
    "plt.clf()\n",
    "#plt.close() # uncomment this if using .py file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate a report with lots of statistical information and export to .txt file\n",
    "# for later reference if needed.\n",
    "\n",
    "# create directory to store stats summaries in\n",
    "if not os.path.exists(\"./statistical summaries/\"):\n",
    "    os.makedirs(\"./statistical summaries/\")\n",
    "\n",
    "file_report = 'statistical summaries/' + ticker_input + \".txt\"\n",
    "with open(file_report, \"w\") as f:\n",
    "    f.write(\"Features shape : {}\".format(df.drop(\"Adj Close\", axis=1).shape))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"Target shape   : {}\".format(df[\"Adj Close\"].shape))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nColumn names\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(df.columns))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nStatistical summary\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(df.describe(include='all')))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nDatatypes\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(df.dtypes))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nPEARSON correlation\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(df.corr(method=\"pearson\")))\n",
    "    f.write(\"\\n\")\n",
    "\n",
    "    f.write(\"\\nMissing Values\")\n",
    "    f.write(\"\\n\")\n",
    "    f.write(str(pd.isnull(df).any()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From here, we can see which features are highly correlated to\n",
    "each other. This is important because if some features are \n",
    "almost identically correlated to eachother, we won't need both\n",
    "of them in our datasets. We will deal with this more during \n",
    "the feature selection phase, but this gives a nice visual\n",
    "of how your feautres correlate to each other. The images created\n",
    "are saved in the project's main directory for your review.\n",
    "\n",
    "Let's create some more visualizations. You can review how to \n",
    "read these plots online and at the links given below :)\n",
    "Then save those plot images too. We won't show them here,\n",
    "just see the saved images in the working directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done visualizing!\n"
     ]
    }
   ],
   "source": [
    "# Univariates\n",
    "sns.set(color_codes=True)\n",
    "colors = [\"y\", \"b\", \"g\", \"r\"]\n",
    "\n",
    "cols = list(df.columns.values)\n",
    "if not os.path.exists(\"./plots/univariate/box\"):\n",
    "    os.makedirs(\"./plots/univariate/box\")\n",
    "\n",
    "# Box plots\n",
    "# https://www.khanacademy.org/math/ap-statistics/summarizing-quantitative-data-ap/stats-box-whisker-plots/v/reading-box-and-whisker-plots\n",
    "for i, col in enumerate(cols):\n",
    "    sns.boxplot(df[col], color=random.choice(colors), orient=\"v\")\n",
    "    plt.savefig(\"./plots/univariate/box/box_\" + str(i) + \".png\")\n",
    "    plt.clf()\n",
    "    plt.close()\n",
    "\n",
    "if not os.path.exists(\"plots/univariate/density\"):\n",
    "    os.makedirs(\"./plots/univariate/density\")\n",
    "\n",
    "# Kernel Density plots\n",
    "# https://en.wikipedia.org/wiki/Kernel_density_estimation\n",
    "for i, col in enumerate(cols):\n",
    "    sns.distplot(df[col], color=random.choice(colors))\n",
    "    plt.savefig(\"./plots/univariate/density/density_\" + str(i) + \".png\")\n",
    "    plt.clf()\n",
    "    plt.close()\n",
    "# Commented out as I get \"LinAlgError: singular matrix\" error after I added in \n",
    "# all those technical indicators.\n",
    "\n",
    "# Bivariates\n",
    "if not os.path.exists(\"./plots/multivariate\"):\n",
    "    os.makedirs(\"./plots/multivariate\")\n",
    "\n",
    "# Scatter plots\n",
    "for i, col in enumerate(cols):\n",
    "    try:\n",
    "        if (i == len(cols) - 1):\n",
    "            pass\n",
    "        else: \n",
    "            sns.jointplot(x=col, y=\"Adj Close\", data=df);\n",
    "            plt.savefig(\"./plots/multivariate/target_vs_\" + str(i) + \".png\")\n",
    "            plt.clf()\n",
    "            plt.close()\n",
    "    except:\n",
    "        pass\n",
    "\n",
    "\n",
    "# Pairplot\n",
    "#sns.pairplot(df) \n",
    "#plt.savefig(\"plots/pairplot.png\")\n",
    "#plt.clf()\n",
    "#plt.close()\n",
    "# I've commented this out because trying to create a pairplot\n",
    "# image of a large dataset can take a while and make a big image file, so feel\n",
    "# free to add this back in if you want but expect it to be slow.\n",
    "\n",
    "# An alternative would be...\n",
    "#sns.pairplot(df.sample(1000)) \n",
    "#plt.savefig(\"plots/pairplot.png\")\n",
    "#plt.clf()\n",
    "#plt.close()\n",
    "\n",
    "print('Done visualizing!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. AUTOMATED FEATURE ENGINEERING! WOOO!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be using the Featuretools module for this step of the process.\n",
    "Feature Engineering, in a very basic sense, is the creation of new\n",
    "features, using the current features of your dataset. To do this\n",
    "part manually, data scientists would have to have some domain knowledge\n",
    "of the field they're working with. If you had a dataset of sales numbers\n",
    "and geographical locations, adding another column or feature of say, the ratio\n",
    "of sales per geographical location would show the relationshiop between sales and\n",
    "the location, giving our model more data to work with. Automatic Feature \n",
    "Engineering takes care of this step for us by creating lots of new features\n",
    "using the entire dataset to bring to light hidden relationships between the\n",
    "features, effectively mitigating the need to know \"everything\" about the \n",
    "field you're trying to figure out. More feature engineering can be found here:\n",
    "https://www.analyticsvidhya.com/blog/2018/08/guide-automated-feature-engineering-featuretools-python/\n",
    "\n",
    "In our example, if we have the 9 MA and 20 MA values, using Deep Feature Synthesis,\n",
    "it would create added columns like \"9MA > 20MA\", \"9MA < 20MA\", \"Adj Close > 9MA\",\n",
    "the list can go on and on, as we'll see in a minute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        Date  PrevOpen  PrevHigh  PrevLow  PrevAdjClose      PrevVol  9MA  \\\n",
      "0 2002-07-12      1.15      1.28     1.13          1.27 2,529,800.00 1.16   \n",
      "1 2002-07-15      1.28      1.28     1.21          1.21 2,837,800.00 1.19   \n",
      "2 2002-07-16      1.24      1.25     1.08          1.14 4,239,200.00 1.20   \n",
      "3 2002-07-17      1.15      1.16     1.10          1.13 1,941,800.00 1.20   \n",
      "4 2002-07-18      1.14      1.16     1.08          1.09 1,261,400.00 1.19   \n",
      "\n",
      "   20MA  UpperBB  MidBB     ...       ATR  BETA  LINEARREG  LINEARREG_ANGLE  \\\n",
      "0  1.06     1.29   1.06     ...      0.08  0.60       1.26             1.43   \n",
      "1  1.07     1.31   1.06     ...      0.08  0.31       1.27             1.40   \n",
      "2  1.08     1.31   1.07     ...      0.09  0.46       1.26             1.20   \n",
      "3  1.09     1.32   1.08     ...      0.09  0.05       1.24             0.91   \n",
      "4  1.10     1.31   1.09     ...      0.09  0.05       1.20             0.57   \n",
      "\n",
      "   LINEARREG_SLOPE  STDDEV  TSF  Open  Adj Close  DailyResult  \n",
      "0             0.02    0.05 1.28  1.28       1.21            0  \n",
      "1             0.02    0.04 1.29  1.24       1.14            0  \n",
      "2             0.02    0.05 1.28  1.15       1.13            0  \n",
      "3             0.02    0.05 1.25  1.14       1.09            0  \n",
      "4             0.01    0.06 1.21  1.12       1.15            1  \n",
      "\n",
      "[5 rows x 36 columns]\n"
     ]
    }
   ],
   "source": [
    "# \"Reset\" the index column for Featuretools to use now \"Date\" column as the index, then we'll revert\n",
    "# the Date column back to an index after feature engineering is done (creating a new, bigger dataset).\n",
    "# But first, save the date column for later re-instating.\n",
    "\n",
    "# These if statements are a way of preventing over additions of indexes when re-running the \n",
    "# notebook. Just ignore it and move on.\n",
    "if \"level_0\" in df:\n",
    "    df = df.drop('level_0',axis=1)\n",
    "if \"index\" in df:\n",
    "    df = df.drop('index',axis=1)\n",
    "if 'Date' in df:\n",
    "    date_values = df['Date']\n",
    "df = df.reset_index()\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216, got 192\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__\n",
      "  return f(*args, **kwds)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Adding the engineered features to the dataframe. This may take a hot minute...\n",
      "Built 2145 features\n",
      "Elapsed: 04:00 | Remaining: 00:00 | Progress: 100%|████████████████████████████████████████| Calculated: 101/101 chunks\n"
     ]
    }
   ],
   "source": [
    "# Let's do it! Auto Feature Engineering using Feature Tools\n",
    "import featuretools as ft\n",
    "#print(ft.list_primitives().to_string()) # To get full list of primitives that could be used\n",
    "\n",
    "print('Adding the engineered features to the dataframe. This may take a hot minute...')\n",
    "\n",
    "# Define the function. We do this in case there's a runtime error, it won't\n",
    "# start the program over again, just stays within the function\n",
    "def feature_engineering_dataset(df):\n",
    "    es = ft.EntitySet(id = 'stockdata')\n",
    "\n",
    "# Now we make sure that the dataframe we feed to the featuretools doesn't include\n",
    "# our label. Notice the drop? Because we don't want to engineer features using the\n",
    "# label that hasn't happened yet, right?\n",
    "    adj_close = list(df['Adj Close'])\n",
    "    dailyresult = list(df['DailyResult'])\n",
    "    dataframe = df.drop(['Adj Close', 'DailyResult'],axis=1)\n",
    "    es.entity_from_dataframe(entity_id = 'data', dataframe = dataframe, index = 'Date')\n",
    "\n",
    "# Pesky warnings\n",
    "    warnings.filterwarnings(\"ignore\", category=RuntimeWarning) \n",
    "    warnings.filterwarnings(\"once\", category=ImportWarning)\n",
    "\n",
    "# Run deep feature synthesis with transformation primitives\n",
    "# Depending on how many features are being added to the dataset,\n",
    "# you may need to tweak the chunk_size and n_jobs variables below.\n",
    "# n_jobs should be tweaked for how many processors are on your\n",
    "# machine, and the chunk_size should be big enough to run quickly,\n",
    "# but small enough that each chunk can fit into memory. See the \n",
    "# Featuretools website for more information on this.\n",
    "    feature_matrix, feature_defs = ft.dfs(n_jobs=1,entityset = es, target_entity = 'data', \n",
    "                                           chunk_size=0.01,max_depth=2,verbose=True,\n",
    "                    #agg_primitives = ['avg_time_between',\n",
    "                     #   'trend','std',],    # review \"print(ft.list_primitives().to_string())\" for more\n",
    "                    trans_primitives = [\n",
    "                        'less_than_equal_to',\n",
    "                        'greater_than_equal_to'\n",
    "                    ])\n",
    "    warnings.filterwarnings(\"once\", category=RuntimeWarning) \n",
    "\n",
    "# Now re-add the Adj Close column because featuretools...\n",
    "    feature_matrix['Adj Close'] = adj_close\n",
    "    feature_matrix['DailyResult'] = dailyresult\n",
    "\n",
    "# Return our new dataset!\n",
    "    return(feature_matrix)\n",
    "\n",
    "# Now run that defined function\n",
    "df = feature_engineering_dataset(df)\n",
    "\n",
    "# Now save a new .csv file with our new features in it for review later if we want\n",
    "if not os.path.exists(\"./historical data/\"):\n",
    "    os.makedirs(\"./historical data/\")\n",
    "dataset.to_csv('./historical data/' + ticker_input + '_with_feature_matrix.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            PrevOpen  PrevHigh  PrevLow  PrevAdjClose      PrevVol  9MA  20MA  \\\n",
      "Date                                                                            \n",
      "2002-07-12      1.15      1.28     1.13          1.27 2,529,800.00 1.16  1.06   \n",
      "2002-07-15      1.28      1.28     1.21          1.21 2,837,800.00 1.19  1.07   \n",
      "2002-07-16      1.24      1.25     1.08          1.14 4,239,200.00 1.20  1.08   \n",
      "2002-07-17      1.15      1.16     1.10          1.13 1,941,800.00 1.20  1.09   \n",
      "2002-07-18      1.14      1.16     1.08          1.09 1,261,400.00 1.19  1.10   \n",
      "\n",
      "            UpperBB  MidBB  LowerBB     ...       PrevHigh >= LowerBB  \\\n",
      "Date                                    ...                             \n",
      "2002-07-12     1.29   1.06     0.82     ...                      True   \n",
      "2002-07-15     1.31   1.06     0.82     ...                      True   \n",
      "2002-07-16     1.31   1.07     0.83     ...                      True   \n",
      "2002-07-17     1.32   1.08     0.85     ...                      True   \n",
      "2002-07-18     1.31   1.09     0.87     ...                      True   \n",
      "\n",
      "            SAR >= LINEARREG_SLOPE  9MA >= ULTOSC  PrevAdjClose >= UpperBB  \\\n",
      "Date                                                                         \n",
      "2002-07-12                    True          False                    False   \n",
      "2002-07-15                    True          False                    False   \n",
      "2002-07-16                    True          False                    False   \n",
      "2002-07-17                    True          False                    False   \n",
      "2002-07-18                    True          False                    False   \n",
      "\n",
      "            MACD >= UpperBB  UpperBB >= PrevVol  TSF >= 20MA  \\\n",
      "Date                                                           \n",
      "2002-07-12            False               False         True   \n",
      "2002-07-15            False               False         True   \n",
      "2002-07-16            False               False         True   \n",
      "2002-07-17            False               False         True   \n",
      "2002-07-18            False               False         True   \n",
      "\n",
      "            MACDSIGNAL >= TSF  Adj Close  DailyResult  \n",
      "Date                                                   \n",
      "2002-07-12              False       1.21            0  \n",
      "2002-07-15              False       1.14            0  \n",
      "2002-07-16              False       1.13            0  \n",
      "2002-07-17              False       1.09            0  \n",
      "2002-07-18              False       1.15            1  \n",
      "\n",
      "[5 rows x 2147 columns]\n",
      "Behold our new dataset! Huzzah! (Remember, theres not really 5 rows, we just printed the head.)\n"
     ]
    }
   ],
   "source": [
    "print(df.head())\n",
    "print('Behold our new dataset! Huzzah! (Remember, theres not really 5 rows, we just printed the head.)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. AUTOMATED FEATURE SELECTION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, as you'll see, we have way more columns in our dataset than when we \n",
    "first started. Surely we don't need ALL of these columns before training\n",
    "our model do we? Nope!\n",
    "\n",
    "So what we'll do now is utilize the Feature-Selector module to help us\n",
    "perform automated feature selection. What's going to happen here is,\n",
    "it's going to find features that have more than 60% missing values,\n",
    "have a correlation threshold of more than 98% (remember our correlation\n",
    "heatmap from before?), and maybe most importantly, get rid of any \n",
    "features that don't add up to a max feature importance of 99%! This uses\n",
    "lightgbm to train a quick model to determine which features are the least\n",
    "important, then get rid of them, leaving us with an awesome dataset!\n",
    "\n",
    "https://github.com/WillKoehrsen/feature-selector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\lightgbm\\__init__.py:27: ResourceWarning: unclosed file <_io.TextIOWrapper name='C:\\\\Users\\\\windowshopr\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python36\\\\lib\\\\site-packages\\\\lightgbm\\\\VERSION.txt' mode='r' encoding='cp1252'>\n",
      "  __version__ = open(os.path.join(dir_path, 'VERSION.txt')).read().strip()\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 features with greater than 0.60 missing values.\n",
      "\n",
      "704 features with a single unique value.\n",
      "\n",
      "1192 features with a correlation magnitude greater than 0.98.\n",
      "\n",
      "Training Gradient Boosting Model\n",
      "\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[41]\tvalid_0's auc: 0.533937\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[8]\tvalid_0's auc: 0.524578\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[3]\tvalid_0's auc: 0.532763\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[4]\tvalid_0's auc: 0.525532\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[1]\tvalid_0's auc: 0.507496\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[7]\tvalid_0's auc: 0.513732\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[6]\tvalid_0's auc: 0.48885\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[1]\tvalid_0's auc: 0.52558\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[3]\tvalid_0's auc: 0.532032\n",
      "Training until validation scores don't improve for 100 rounds.\n",
      "Early stopping, best iteration is:\n",
      "[4]\tvalid_0's auc: 0.522987\n",
      "\n",
      "1962 features with zero importance after one-hot encoding.\n",
      "\n",
      "154 features required for cumulative importance of 0.99 after one hot encoding.\n",
      "1986 features do not contribute to cumulative importance of 0.99.\n",
      "\n",
      "2034 total features out of 2140 identified for removal after one-hot encoding.\n",
      "\n",
      "Total of 2034 features identified for removal\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['PrevVol <= TSF', 'OBV >= STDDEV', '20MA >= STOCHSLOWD', 'STDDEV <= MACD', 'MOM >= DX', 'LowerBB <= MidBB', 'LINEARREG_ANGLE >= UpperBB', 'APO >= PrevAdjClose', 'PrevAdjClose <= MACDHIST', 'MOM <= 20MA', 'Open', 'Open >= PrevVol', 'ADX <= STOCHSLOWD', 'APO >= CCI', 'APO >= MACDHIST', 'MACD >= ADX', 'STOCHSLOWK <= MACD', 'BETA <= OBV', 'PrevHigh <= APO', 'PrevLow <= PrevHigh', 'MACD >= MACDHIST', 'PrevAdjClose >= 20MA', '20MA <= DX', '20MA >= PrevVol', '9MA >= PrevLow', 'Open <= ATR', 'LINEARREG_ANGLE <= STDDEV', 'LINEARREG_SLOPE >= PrevAdjClose', 'ADX >= LINEARREG', 'DX >= Open', 'DX >= ULTOSC', 'MACD >= DX', 'ROC >= BETA', 'LINEARREG >= MidBB', 'UpperBB <= ADX', '9MA >= MACDHIST', 'ULTOSC <= ROC', 'STOCHSLOWD >= MACDSIGNAL', 'LowerBB <= BETA', 'MACD >= TSF', 'STDDEV >= TSF', 'SAR >= MACDHIST', 'DX >= BETA', 'ULTOSC <= SAR', 'SAR <= OBV', 'STOCHSLOWK <= MOM', 'UpperBB <= MACDSIGNAL', 'Open >= LINEARREG', 'LowerBB <= PrevVol', 'PrevVol <= MACDSIGNAL', 'CCI >= LINEARREG_SLOPE', 'UpperBB >= ULTOSC', 'PrevOpen >= STDDEV', 'MACD >= UpperBB', 'BETA <= MidBB', 'Open >= SAR', 'LINEARREG_ANGLE <= LINEARREG', 'PrevLow <= 20MA', 'LINEARREG >= PrevVol', 'ADX <= STOCHSLOWK', 'MACDSIGNAL >= MACDHIST', 'RSI >= DX', 'MACDHIST <= STOCHSLOWD', 'UpperBB', 'PrevHigh <= DX', 'LowerBB <= 9MA', 'STOCHSLOWK >= LINEARREG_SLOPE', '20MA >= TSF', 'PrevHigh >= RSI', 'MidBB >= 20MA', 'PrevLow >= Open', 'OBV >= MACD', 'PrevOpen <= LINEARREG_ANGLE', 'ATR >= ADX', 'SAR <= DX', 'MidBB <= BETA', 'LINEARREG_ANGLE <= PrevLow', 'LINEARREG_SLOPE', 'PrevLow <= ROC', 'OBV >= UpperBB', 'CCI <= SAR', 'PrevAdjClose >= OBV', 'ULTOSC >= MACD', 'OBV <= PrevAdjClose', 'STDDEV >= STOCHSLOWK', 'RSI >= ATR', 'STOCHSLOWD <= STDDEV', 'ATR >= PrevOpen', 'CCI <= Open', 'STOCHSLOWK <= TSF', 'ATR >= Open', 'STOCHSLOWD >= OBV', 'APO >= LowerBB', 'OBV <= PrevVol', 'MACDHIST >= ULTOSC', 'ATR >= MidBB', 'STOCHSLOWK <= PrevAdjClose', 'Open <= MACDSIGNAL', 'MACDHIST >= TSF', 'MACDSIGNAL >= TSF', 'PrevAdjClose >= MACDSIGNAL', 'MOM <= PrevOpen', 'MACDSIGNAL <= MACD', 'PrevOpen <= TSF', 'STOCHSLOWD <= LINEARREG_SLOPE', 'LINEARREG_SLOPE <= MOM', 'ATR <= STOCHSLOWD', 'STOCHSLOWD >= LINEARREG', 'TSF >= LINEARREG', 'LINEARREG >= Open', 'PrevOpen <= PrevLow', 'PrevLow <= 9MA', 'MidBB >= DX', 'APO <= LowerBB', 'MACDSIGNAL <= OBV', 'MOM >= PrevHigh', 'PrevAdjClose <= APO', 'MACDSIGNAL >= PrevHigh', 'PrevOpen >= PrevVol', 'MOM >= 20MA', 'PrevLow <= STOCHSLOWK', 'ATR <= MidBB', 'Open >= 20MA', 'LINEARREG_SLOPE >= APO', 'LowerBB >= UpperBB', 'PrevOpen >= CCI', 'MACDHIST >= LINEARREG_ANGLE', 'SAR >= APO', 'LowerBB <= LINEARREG', 'MACDSIGNAL >= Open', 'OBV >= PrevLow', 'SAR >= LowerBB', 'MACDHIST <= LINEARREG_ANGLE', 'CCI >= 9MA', '20MA <= MACDSIGNAL', 'OBV >= LowerBB', 'STOCHSLOWK >= PrevLow', 'MOM <= STDDEV', 'STDDEV >= STOCHSLOWD', 'BETA >= MACDHIST', 'RSI <= ROC', 'LINEARREG <= RSI', 'STOCHSLOWK <= UpperBB', 'STDDEV >= MidBB', 'SAR <= MidBB', 'OBV >= ADX', 'ULTOSC <= UpperBB', 'MidBB >= LowerBB', '20MA >= MACDHIST', 'TSF >= 9MA', 'ROC >= SAR', 'APO <= OBV', 'UpperBB >= SAR', 'MOM >= ROC', 'ULTOSC >= APO', 'LINEARREG_SLOPE >= LowerBB', 'MACDSIGNAL <= MidBB', 'PrevAdjClose >= APO', 'ROC <= PrevAdjClose', 'ROC >= STDDEV', 'MACDHIST <= UpperBB', 'STDDEV <= PrevAdjClose', 'MACD <= CCI', 'LowerBB <= CCI', 'PrevVol >= APO', 'CCI >= STOCHSLOWK', 'STOCHSLOWK <= ROC', 'ULTOSC <= DX', 'MOM <= ULTOSC', 'ULTOSC >= MACDHIST', 'LINEARREG_SLOPE <= STOCHSLOWD', 'PrevAdjClose <= STDDEV', 'LINEARREG_ANGLE >= MACD', 'PrevHigh <= LowerBB', 'Open <= PrevVol', 'ADX <= LowerBB', 'MidBB >= PrevHigh', 'STOCHSLOWD <= UpperBB', 'SAR <= 9MA', 'ULTOSC >= SAR', 'Open >= STDDEV', '9MA >= MidBB', 'LINEARREG_ANGLE >= ROC', 'PrevHigh >= MACDHIST', 'ULTOSC >= ATR', 'ROC <= MACD', 'LINEARREG_SLOPE >= MACDHIST', 'LINEARREG <= OBV', 'STOCHSLOWD <= ADX', 'LINEARREG_SLOPE <= MidBB', 'STOCHSLOWK <= PrevOpen', 'LINEARREG >= MACDSIGNAL', 'STDDEV >= MACDHIST', 'MidBB >= APO', 'LINEARREG <= ADX', '9MA <= MACDHIST', 'LowerBB <= LINEARREG_ANGLE', 'RSI >= ROC', 'ATR <= ADX', 'ATR <= MOM', 'PrevOpen <= ATR', 'ATR <= LINEARREG_SLOPE', 'PrevVol >= PrevHigh', 'MACD >= MACDSIGNAL', 'ULTOSC <= BETA', 'STOCHSLOWK <= 20MA', 'DX <= MACDHIST', 'UpperBB >= ROC', 'MidBB <= UpperBB', 'RSI >= PrevHigh', 'LINEARREG_ANGLE >= PrevHigh', 'ULTOSC <= 20MA', 'ADX >= PrevLow', 'LowerBB >= MACDHIST', 'LowerBB >= RSI', 'LowerBB >= STOCHSLOWD', 'UpperBB >= LINEARREG', 'PrevHigh >= 9MA', 'DX >= STOCHSLOWD', 'PrevVol <= LINEARREG', 'MACDSIGNAL >= 20MA', 'OBV <= ROC', 'ULTOSC <= MACDSIGNAL', 'MOM <= 9MA', 'OBV <= MACDHIST', 'PrevOpen <= PrevHigh', 'PrevLow >= PrevHigh', 'DX >= PrevOpen', 'LINEARREG <= DX', 'ROC >= MACD', '20MA >= MidBB', 'SAR <= TSF', 'MACDHIST >= APO', 'DX <= 20MA', 'ULTOSC >= STDDEV', 'SAR <= ADX', 'ATR >= PrevAdjClose', 'OBV >= PrevOpen', '9MA <= PrevVol', 'STOCHSLOWD >= DX', '20MA <= CCI', 'TSF <= 9MA', 'STOCHSLOWK >= MidBB', 'MACDHIST >= ATR', 'LINEARREG_ANGLE >= PrevLow', 'ROC <= STOCHSLOWK', 'PrevAdjClose <= STOCHSLOWD', 'STDDEV >= MOM', 'PrevOpen <= RSI', 'RSI <= LINEARREG_SLOPE', 'MACD <= MidBB', 'MidBB <= 9MA', 'PrevLow <= ADX', 'RSI >= MOM', 'LINEARREG_ANGLE >= PrevOpen', 'ADX <= LINEARREG_ANGLE', 'MOM >= MACD', 'ADX >= ULTOSC', 'LINEARREG_ANGLE >= 20MA', 'LINEARREG <= PrevLow', 'LINEARREG_ANGLE <= STOCHSLOWD', 'PrevLow >= 9MA', 'TSF <= Open', 'RSI <= LowerBB', 'LowerBB >= ATR', 'PrevLow >= MACD', 'DX >= SAR', 'RSI >= LINEARREG_ANGLE', 'LINEARREG_SLOPE <= 20MA', 'CCI <= LINEARREG_SLOPE', 'RSI <= MACDSIGNAL', 'TSF <= DX', 'STOCHSLOWK >= RSI', 'DX <= LINEARREG_ANGLE', 'STOCHSLOWK >= ATR', 'ADX <= BETA', 'ADX >= Open', 'STOCHSLOWD <= MACD', 'PrevLow >= STOCHSLOWK', 'TSF >= UpperBB', 'STOCHSLOWK <= ULTOSC', 'ROC <= DX', 'DX <= MidBB', 'ROC <= MACDHIST', 'ROC <= Open', 'SAR <= PrevLow', 'PrevLow <= STOCHSLOWD', 'BETA >= 9MA', 'CCI <= LowerBB', '9MA >= LINEARREG', 'LowerBB <= PrevLow', 'CCI <= MidBB', 'PrevAdjClose <= ATR', 'ULTOSC >= MOM', 'MidBB <= PrevVol', '20MA <= MACD', 'PrevHigh >= PrevLow', 'ATR >= LINEARREG_SLOPE', 'ULTOSC >= CCI', 'LINEARREG_SLOPE <= ATR', 'ULTOSC >= BETA', 'MOM >= UpperBB', 'BETA >= APO', 'ULTOSC >= ADX', 'LINEARREG_SLOPE >= STOCHSLOWK', 'MACD <= BETA', 'UpperBB >= CCI', 'DX >= PrevAdjClose', 'PrevAdjClose >= MOM', 'STOCHSLOWK <= ATR', 'STDDEV <= APO', 'PrevHigh <= PrevAdjClose', 'LINEARREG_ANGLE >= BETA', 'LINEARREG_ANGLE >= LINEARREG', 'PrevOpen <= SAR', '9MA >= OBV', 'LINEARREG_SLOPE >= MidBB', 'MACD >= PrevVol', 'DX <= LINEARREG_SLOPE', 'MidBB >= MOM', 'UpperBB >= PrevHigh', 'LowerBB >= LINEARREG_SLOPE', 'SAR <= MOM', 'CCI >= ADX', 'PrevAdjClose <= LINEARREG_SLOPE', 'UpperBB <= MACDHIST', 'ADX >= ROC', 'LowerBB <= STDDEV', 'ATR >= PrevVol', 'TSF >= Open', 'PrevVol >= MACDHIST', 'SAR >= STOCHSLOWD', 'STOCHSLOWK <= LINEARREG', 'CCI >= MACDHIST', 'ADX <= RSI', 'ROC <= MidBB', 'Open <= APO', 'PrevHigh >= DX', 'MACDHIST >= OBV', 'LowerBB >= PrevOpen', 'MACDHIST <= Open', 'LowerBB >= MACD', 'STDDEV <= ADX', 'PrevLow >= PrevVol', 'MACDSIGNAL >= APO', 'STDDEV <= LINEARREG', 'MidBB >= 9MA', '20MA <= LowerBB', 'ULTOSC >= 20MA', 'ROC >= CCI', 'TSF >= MACDHIST', 'OBV <= ATR', 'OBV <= 9MA', 'STOCHSLOWK >= TSF', 'MACDHIST <= SAR', '20MA >= PrevLow', 'ATR <= RSI', 'MACDHIST >= LINEARREG_SLOPE', 'MOM >= LINEARREG_SLOPE', 'PrevLow >= ATR', 'MACDSIGNAL >= ULTOSC', 'PrevAdjClose <= SAR', 'STOCHSLOWK <= APO', 'Open <= LINEARREG_ANGLE', 'BETA <= LINEARREG_ANGLE', 'PrevAdjClose <= ULTOSC', '20MA <= PrevHigh', 'ULTOSC >= TSF', 'PrevOpen <= MidBB', 'MACDSIGNAL <= 9MA', 'MACDHIST <= STOCHSLOWK', 'PrevLow <= LowerBB', 'UpperBB <= 9MA', 'MidBB <= MACD', 'STOCHSLOWD >= LINEARREG_ANGLE', 'LINEARREG_SLOPE <= PrevHigh', 'MACDSIGNAL >= SAR', 'CCI <= ROC', 'DX <= ROC', '9MA >= APO', 'STOCHSLOWK >= STDDEV', 'STDDEV >= MACDSIGNAL', '9MA <= PrevHigh', 'Open <= SAR', 'PrevVol >= LINEARREG', 'MACDHIST >= CCI', 'STOCHSLOWK >= MACD', '20MA >= BETA', 'RSI <= ADX', 'LINEARREG <= MOM', 'SAR <= APO', 'CCI <= STOCHSLOWD', 'STOCHSLOWD <= LINEARREG_ANGLE', 'BETA >= STOCHSLOWK', 'MACDHIST >= 9MA', 'ULTOSC <= ADX', 'UpperBB <= STDDEV', 'PrevVol >= ADX', '9MA >= PrevOpen', 'LINEARREG <= PrevOpen', 'ULTOSC <= ATR', 'TSF >= MACD', 'PrevHigh <= BETA', 'OBV >= LINEARREG_ANGLE', 'MACD <= 9MA', 'PrevAdjClose <= MidBB', 'PrevOpen <= MACDSIGNAL', 'LINEARREG_ANGLE <= MACDHIST', 'LINEARREG <= APO', '9MA <= MOM', 'TSF <= APO', 'LINEARREG <= ATR', 'MACDSIGNAL <= LowerBB', 'OBV <= PrevLow', 'CCI <= PrevHigh', 'ULTOSC <= STOCHSLOWD', 'APO >= 9MA', 'BETA >= TSF', 'MidBB >= OBV', 'MACD <= LINEARREG', '9MA <= DX', 'Open <= 9MA', 'OBV >= STOCHSLOWK', 'PrevVol <= PrevHigh', 'PrevAdjClose <= LINEARREG_ANGLE', 'ULTOSC <= MidBB', 'LINEARREG_SLOPE >= CCI', 'STOCHSLOWK >= LINEARREG', 'BETA >= PrevVol', 'Open >= ULTOSC', 'OBV >= PrevHigh', 'PrevLow >= ROC', 'LINEARREG_ANGLE <= PrevOpen', 'Open >= STOCHSLOWK', 'OBV >= PrevVol', 'RSI >= BETA', 'OBV <= LINEARREG', 'DX <= Open', 'MACDSIGNAL >= BETA', 'PrevVol <= ADX', 'DX <= OBV', 'TSF <= ATR', 'PrevLow <= STDDEV', 'TSF <= LINEARREG_ANGLE', 'UpperBB <= LowerBB', 'ROC >= PrevOpen', '20MA <= MACDHIST', 'Open <= RSI', 'RSI <= TSF', 'LINEARREG_ANGLE >= STOCHSLOWK', 'RSI <= BETA', 'PrevAdjClose >= LINEARREG', 'LINEARREG_ANGLE >= ATR', 'PrevLow >= ADX', 'ULTOSC <= Open', 'RSI >= MidBB', 'ROC >= ULTOSC', 'PrevHigh >= PrevVol', 'STOCHSLOWD >= 9MA', 'ADX >= STDDEV', 'STOCHSLOWK <= MACDHIST', 'APO >= ROC', 'RSI <= ULTOSC', 'SAR <= MACDSIGNAL', 'MACDSIGNAL <= MACDHIST', 'TSF >= STOCHSLOWD', 'ADX >= BETA', 'Open <= OBV', 'STOCHSLOWK >= PrevHigh', 'LINEARREG_ANGLE <= CCI', 'TSF <= UpperBB', 'STOCHSLOWD >= BETA', 'OBV <= SAR', 'PrevVol <= LINEARREG_SLOPE', 'PrevAdjClose >= PrevVol', 'PrevVol >= MOM', 'PrevOpen >= PrevHigh', 'MidBB >= UpperBB', 'RSI <= MOM', 'PrevVol <= ULTOSC', 'PrevHigh >= SAR', 'MACDSIGNAL <= RSI', 'APO >= PrevVol', 'BETA <= SAR', '20MA <= PrevLow', 'MACDSIGNAL >= LINEARREG_ANGLE', 'LINEARREG_SLOPE <= LowerBB', 'PrevHigh >= LINEARREG_SLOPE', 'ATR <= STOCHSLOWK', 'MOM <= MACDHIST', 'MACD <= PrevLow', 'MOM >= Open', 'PrevOpen <= Open', 'PrevHigh <= MOM', 'STOCHSLOWD >= STOCHSLOWK', '9MA >= MACD', 'LowerBB', 'UpperBB <= PrevHigh', 'LINEARREG_SLOPE <= ADX', 'ROC <= ADX', 'ADX >= MOM', 'ATR <= PrevAdjClose', 'ROC <= RSI', 'ATR >= PrevLow', 'PrevHigh <= STDDEV', '20MA >= MACD', 'SAR >= MACD', 'ULTOSC <= PrevAdjClose', 'ADX <= MidBB', 'Open >= STOCHSLOWD', 'MACDHIST >= BETA', 'MACDHIST <= PrevAdjClose', 'Open >= RSI', 'ADX <= PrevHigh', 'APO <= MACDHIST', 'OBV >= MidBB', 'PrevAdjClose <= MOM', 'PrevAdjClose >= PrevLow', 'UpperBB >= ATR', 'STOCHSLOWK <= MidBB', 'LINEARREG_ANGLE <= MOM', 'Open <= ROC', 'SAR >= LINEARREG_ANGLE', 'SAR >= PrevOpen', 'SAR <= ULTOSC', 'MACD >= OBV', 'APO <= 20MA', 'LINEARREG_SLOPE >= MOM', 'ADX >= TSF', 'UpperBB >= PrevLow', 'STDDEV >= PrevHigh', 'UpperBB <= BETA', 'PrevVol >= ATR', 'UpperBB >= MidBB', 'PrevOpen <= STDDEV', 'STOCHSLOWD <= ROC', '9MA <= LINEARREG_SLOPE', 'LowerBB <= RSI', '9MA >= BETA', 'OBV >= Open', 'DX >= PrevLow', 'LINEARREG >= SAR', 'BETA <= LowerBB', 'MACDHIST <= 20MA', 'Open <= MOM', 'TSF >= PrevLow', 'PrevVol >= PrevLow', 'STOCHSLOWK >= LINEARREG_ANGLE', 'LINEARREG_ANGLE <= 9MA', 'MACD <= RSI', 'LINEARREG >= LINEARREG_ANGLE', 'STOCHSLOWD <= PrevLow', 'LINEARREG_SLOPE <= MACDHIST', 'STDDEV >= DX', 'LINEARREG >= ROC', 'OBV <= DX', 'PrevVol <= STOCHSLOWD', 'UpperBB <= 20MA', 'SAR >= LINEARREG', 'PrevAdjClose <= MACD', 'STOCHSLOWD >= LowerBB', 'TSF >= 20MA', 'PrevOpen >= SAR', 'ULTOSC >= MACDSIGNAL', 'MidBB <= STDDEV', 'OBV <= UpperBB', 'LINEARREG <= ULTOSC', '20MA >= ADX', 'Open <= STDDEV', 'APO >= STOCHSLOWD', 'ATR >= MACDHIST', 'PrevOpen >= STOCHSLOWK', 'ADX <= STDDEV', 'MOM >= STOCHSLOWD', 'MACDHIST >= Open', 'MOM <= Open', 'TSF >= MidBB', 'PrevOpen >= ADX', 'APO >= MACDSIGNAL', 'STDDEV >= RSI', 'STOCHSLOWK <= PrevHigh', 'PrevLow >= LINEARREG_SLOPE', 'PrevHigh >= MACD', 'LINEARREG >= STOCHSLOWK', 'DX <= TSF', 'TSF <= PrevAdjClose', '9MA <= ADX', 'DX >= UpperBB', 'APO >= STOCHSLOWK', 'LINEARREG_SLOPE >= PrevVol', 'LINEARREG >= PrevHigh', 'ADX >= LowerBB', 'DX >= LowerBB', 'ATR >= SAR', 'ADX >= ATR', 'LINEARREG_ANGLE >= ADX', 'BETA <= APO', 'PrevAdjClose <= CCI', 'MACDSIGNAL >= PrevOpen', '9MA >= STOCHSLOWD', 'LINEARREG >= ULTOSC', 'PrevOpen <= MOM', 'PrevVol <= LINEARREG_ANGLE', 'MACDHIST <= TSF', 'MACDSIGNAL <= CCI', 'LINEARREG_SLOPE <= APO', 'ULTOSC <= PrevVol', 'MOM >= PrevAdjClose', 'STOCHSLOWD >= ATR', '9MA >= MOM', 'BETA >= ADX', 'LINEARREG_SLOPE <= DX', 'LINEARREG_SLOPE <= LINEARREG', 'ADX <= APO', 'TSF <= ADX', 'TSF >= ADX', 'ROC <= UpperBB', 'STDDEV <= ULTOSC', 'STDDEV <= ATR', 'BETA >= LowerBB', 'MACDHIST <= MidBB', 'STDDEV <= CCI', 'BETA >= RSI', 'STOCHSLOWK >= OBV', 'OBV >= STOCHSLOWD', 'PrevAdjClose >= ULTOSC', 'LINEARREG >= PrevOpen', 'ATR <= TSF', '9MA >= ADX', 'PrevAdjClose >= DX', 'LowerBB >= 20MA', 'LINEARREG_ANGLE >= MOM', 'PrevVol >= LINEARREG_ANGLE', 'ADX >= MACD', 'BETA >= PrevAdjClose', 'STOCHSLOWK <= Open', 'PrevOpen >= 20MA', 'TSF >= STDDEV', 'MACD >= Open', 'MACD >= ATR', 'STOCHSLOWD <= MACDHIST', 'SAR <= MACDHIST', 'LowerBB >= MOM', 'MACDHIST >= MACDSIGNAL', 'MidBB >= PrevOpen', 'UpperBB <= STOCHSLOWK', 'MOM >= ATR', 'MidBB <= PrevOpen', 'STOCHSLOWK >= STOCHSLOWD', 'MidBB <= LINEARREG_ANGLE', 'LowerBB <= PrevHigh', 'UpperBB <= LINEARREG_ANGLE', 'SAR <= CCI', 'PrevVol >= LowerBB', 'MidBB >= RSI', 'ATR <= PrevVol', 'LINEARREG >= DX', 'LowerBB >= SAR', 'LINEARREG_SLOPE >= OBV', 'Open <= LowerBB', 'BETA >= LINEARREG', 'UpperBB <= LINEARREG', 'PrevAdjClose <= BETA', 'SAR >= PrevAdjClose', 'LINEARREG_SLOPE >= LINEARREG', 'TSF <= LowerBB', 'MOM <= UpperBB', 'ADX >= PrevVol', '20MA >= PrevOpen', '20MA >= ATR', 'MACD <= PrevOpen', '9MA >= MACDSIGNAL', 'Open >= MACDHIST', 'MOM <= STOCHSLOWK', 'ATR >= STDDEV', 'SAR <= Open', 'Open >= MidBB', 'MACD >= PrevLow', 'ADX <= MACDHIST', '9MA >= 20MA', 'PrevHigh <= 20MA', 'LowerBB >= MACDSIGNAL', 'APO >= BETA', 'ATR >= 9MA', 'CCI <= UpperBB', 'ULTOSC <= STDDEV', 'UpperBB >= 20MA', 'PrevOpen >= 9MA', 'TSF <= RSI', 'MACDHIST <= CCI', 'MACD >= STOCHSLOWK', 'MidBB <= TSF', 'STDDEV >= LINEARREG_ANGLE', 'MidBB >= STDDEV', 'MOM >= PrevVol', 'PrevVol <= APO', '9MA >= LINEARREG_SLOPE', 'LINEARREG <= STOCHSLOWD', 'MidBB >= LINEARREG_ANGLE', 'PrevOpen >= BETA', '20MA >= PrevAdjClose', 'LINEARREG_SLOPE >= STDDEV', 'PrevVol >= MidBB', 'DX <= RSI', 'CCI <= PrevLow', 'UpperBB >= LINEARREG_SLOPE', 'OBV <= MidBB', 'MOM <= OBV', 'STDDEV >= LINEARREG', 'MACD <= LowerBB', 'LINEARREG_SLOPE >= PrevLow', 'STDDEV >= APO', 'PrevHigh <= RSI', 'UpperBB <= LINEARREG_SLOPE', 'ADX >= RSI', 'STOCHSLOWK <= SAR', 'MidBB >= PrevAdjClose', 'LINEARREG_ANGLE >= OBV', 'OBV <= TSF', 'DX >= ADX', 'RSI >= SAR', 'STDDEV <= LINEARREG_SLOPE', 'MidBB <= CCI', 'SAR <= PrevOpen', 'BETA >= OBV', 'ATR <= ULTOSC', 'STOCHSLOWK <= LINEARREG_SLOPE', 'PrevLow <= LINEARREG_SLOPE', 'APO <= MOM', 'PrevHigh >= STOCHSLOWK', 'SAR >= ATR', 'BETA <= ROC', 'LINEARREG_ANGLE >= 9MA', 'PrevOpen <= 20MA', '20MA', 'SAR >= ROC', 'PrevAdjClose <= 9MA', 'STDDEV <= SAR', 'PrevOpen >= OBV', 'ATR >= STOCHSLOWD', 'UpperBB >= MACDHIST', 'MOM <= PrevVol', 'PrevLow <= DX', 'MACD >= MOM', 'UpperBB <= ROC', 'MACD >= MidBB', 'UpperBB >= Open', 'ADX >= PrevHigh', 'TSF >= ROC', 'SAR >= 20MA', 'CCI >= MidBB', 'PrevOpen >= ROC', 'LowerBB >= OBV', 'ULTOSC >= PrevHigh', 'ATR >= DX', 'LINEARREG_ANGLE >= STOCHSLOWD', 'PrevAdjClose >= Open', '9MA >= LINEARREG_ANGLE', 'PrevHigh <= PrevLow', 'RSI <= STOCHSLOWK', 'BETA >= 20MA', 'ROC >= PrevHigh', 'PrevOpen >= STOCHSLOWD', 'MACD <= ATR', 'STDDEV <= UpperBB', 'PrevLow >= MACDHIST', 'LINEARREG >= 20MA', 'STOCHSLOWD >= PrevVol', 'LINEARREG_SLOPE <= LINEARREG_ANGLE', 'BETA >= MOM', 'DX >= PrevVol', 'DX >= ATR', 'LINEARREG_SLOPE >= STOCHSLOWD', 'CCI <= 9MA', 'PrevHigh <= PrevVol', 'PrevHigh <= SAR', 'TSF <= CCI', 'STOCHSLOWD <= RSI', 'STOCHSLOWK >= ADX', 'Open <= DX', '20MA <= BETA', 'PrevLow <= MACDSIGNAL', 'CCI <= MACD', 'PrevLow <= MOM', 'ATR <= PrevHigh', 'DX <= LINEARREG', '20MA <= LINEARREG', 'LowerBB <= ROC', 'MACDSIGNAL <= ATR', 'PrevHigh >= LINEARREG_ANGLE', 'OBV <= STDDEV', 'DX >= MidBB', 'APO <= BETA', 'RSI >= TSF', 'PrevVol >= LINEARREG_SLOPE', 'MACD <= TSF', 'PrevAdjClose >= LowerBB', 'PrevVol >= OBV', '20MA <= LINEARREG_SLOPE', 'MACDHIST >= ROC', 'PrevVol <= RSI', 'PrevAdjClose <= PrevVol', 'BETA <= LINEARREG', 'PrevOpen <= STOCHSLOWK', 'Open <= MidBB', 'TSF >= PrevHigh', 'MidBB <= PrevHigh', 'MOM >= OBV', 'CCI <= MACDSIGNAL', 'LINEARREG_SLOPE >= RSI', 'OBV >= LINEARREG', 'LINEARREG_SLOPE >= TSF', 'PrevAdjClose <= OBV', 'MidBB <= ATR', 'UpperBB >= STDDEV', 'MidBB >= SAR', 'APO <= UpperBB', 'BETA <= STOCHSLOWD', 'RSI >= APO', 'OBV >= 20MA', 'STOCHSLOWK <= BETA', 'ROC <= STDDEV', 'ATR >= OBV', 'PrevAdjClose >= MACDHIST', 'ROC <= ULTOSC', 'PrevLow >= OBV', 'LowerBB >= MidBB', 'Open <= STOCHSLOWD', 'APO <= ROC', 'DX <= PrevAdjClose', 'BETA <= STOCHSLOWK', 'TSF >= MOM', 'MOM <= MACD', '20MA >= LINEARREG_ANGLE', 'MACDSIGNAL <= PrevAdjClose', 'STDDEV >= SAR', 'SAR <= BETA', 'BETA >= MACD', 'MACDSIGNAL >= ATR', 'PrevVol <= MACD', 'ADX <= 9MA', 'LINEARREG_ANGLE <= PrevHigh', 'CCI >= ULTOSC', 'MOM >= MACDHIST', '9MA <= LINEARREG_ANGLE', 'APO >= PrevHigh', 'MidBB <= MACDSIGNAL', 'MACDSIGNAL >= RSI', 'ATR <= CCI', 'ULTOSC >= OBV', 'MACD >= CCI', 'LINEARREG_ANGLE >= MACDSIGNAL', 'MidBB <= ULTOSC', 'PrevHigh >= OBV', 'PrevOpen >= MACD', 'BETA <= STDDEV', '9MA <= LINEARREG', 'OBV <= CCI', 'MACDHIST <= APO', 'ATR <= UpperBB', 'APO <= STOCHSLOWK', 'STOCHSLOWD <= SAR', 'PrevAdjClose >= PrevHigh', 'PrevAdjClose <= UpperBB', 'STDDEV <= PrevHigh', 'CCI <= PrevOpen', 'DX <= CCI', 'APO >= PrevOpen', 'RSI >= PrevVol', 'MidBB >= MACD', 'PrevLow <= MACDHIST', 'PrevVol >= RSI', 'CCI >= ROC', 'MACDHIST >= PrevLow', 'STOCHSLOWD <= APO', 'PrevAdjClose <= 20MA', 'LINEARREG >= CCI', 'PrevAdjClose >= ADX', 'BETA <= DX', 'STOCHSLOWD <= PrevOpen', 'RSI >= STOCHSLOWK', 'MACDSIGNAL >= STOCHSLOWD', 'DX >= LINEARREG_SLOPE', 'SAR <= LowerBB', 'LowerBB >= PrevLow', 'MidBB >= STOCHSLOWD', '9MA <= MACDSIGNAL', 'BETA >= CCI', 'PrevVol >= Open', 'ADX <= UpperBB', 'OBV <= STOCHSLOWK', 'DX <= MACD', 'LINEARREG_ANGLE >= PrevAdjClose', 'MidBB >= BETA', 'PrevLow >= STOCHSLOWD', 'SAR <= RSI', 'SAR >= MACDSIGNAL', 'DX >= PrevHigh', 'PrevOpen >= APO', 'MACDSIGNAL >= STOCHSLOWK', 'STOCHSLOWD >= PrevHigh', 'MACDSIGNAL <= PrevLow', 'LINEARREG_ANGLE >= TSF', 'PrevVol >= MACD', '20MA <= ULTOSC', 'LINEARREG >= UpperBB', '20MA <= ADX', 'MOM <= PrevLow', 'PrevOpen <= UpperBB', 'PrevLow <= PrevVol', 'SAR >= LINEARREG_SLOPE', 'ROC <= CCI', 'STDDEV <= OBV', 'OBV <= MACD', 'APO <= LINEARREG_ANGLE', 'MACDHIST >= 20MA', 'LINEARREG_ANGLE <= ATR', 'STDDEV <= STOCHSLOWD', 'STDDEV >= OBV', 'CCI <= PrevAdjClose', 'MACD >= PrevHigh', 'STDDEV <= PrevOpen', 'PrevVol <= PrevAdjClose', 'STOCHSLOWD >= MACDHIST', 'Open >= UpperBB', 'STOCHSLOWD >= TSF', 'UpperBB >= MACD', 'STDDEV <= MidBB', 'Open >= 9MA', 'Open <= CCI', 'ADX >= 9MA', 'PrevHigh >= 20MA', 'OBV <= MOM', '20MA >= STDDEV', 'PrevHigh >= STOCHSLOWD', 'LINEARREG <= STOCHSLOWK', 'CCI >= 20MA', 'PrevOpen >= DX', 'TSF <= MACD', 'ULTOSC <= APO', 'RSI <= MidBB', 'ULTOSC <= MACDHIST', 'PrevLow <= ATR', 'MACDHIST >= STDDEV', 'APO <= Open', 'ROC >= LINEARREG', 'STDDEV >= ULTOSC', 'OBV <= LINEARREG_ANGLE', 'ATR >= 20MA', 'CCI >= PrevHigh', 'PrevHigh <= OBV', 'MACD >= LINEARREG_ANGLE', 'PrevLow <= SAR', 'PrevLow >= CCI', 'STOCHSLOWD >= 20MA', 'LINEARREG_SLOPE <= BETA', 'SAR <= STDDEV', 'MACD <= ROC', 'STOCHSLOWK >= PrevVol', 'PrevLow >= ULTOSC', 'ADX >= STOCHSLOWK', 'RSI <= 20MA', 'STDDEV <= BETA', 'MACDSIGNAL >= ADX', 'BETA <= CCI', '9MA <= MACD', 'PrevHigh <= STOCHSLOWD', 'MOM >= STOCHSLOWK', 'PrevAdjClose >= 9MA', '20MA >= LINEARREG', 'SAR <= ATR', 'Open <= ADX', 'ROC >= TSF', 'LINEARREG <= MACD', 'MACDSIGNAL <= SAR', '9MA <= OBV', 'LowerBB >= CCI', 'LINEARREG_ANGLE <= DX', '20MA >= OBV', 'TSF <= MACDSIGNAL', 'LINEARREG_SLOPE >= LINEARREG_ANGLE', 'SAR >= PrevLow', 'PrevLow >= BETA', 'ROC >= PrevVol', 'CCI <= DX', 'ATR >= UpperBB', 'STOCHSLOWK >= 20MA', 'STOCHSLOWD >= PrevOpen', 'TSF <= ROC', 'CCI <= TSF', 'OBV <= BETA', 'ROC <= TSF', 'LINEARREG_ANGLE <= MidBB', 'LINEARREG_ANGLE <= OBV', 'DX >= 9MA', 'Open >= MACDSIGNAL', 'MACDHIST <= PrevOpen', '20MA <= OBV', 'PrevHigh >= ADX', 'PrevOpen <= DX', 'RSI <= PrevVol', '20MA >= LINEARREG_SLOPE', 'MidBB <= ROC', 'ULTOSC >= Open', 'APO >= LINEARREG_ANGLE', 'SAR <= 20MA', 'MACDSIGNAL >= UpperBB', 'LINEARREG >= 9MA', 'UpperBB >= STOCHSLOWD', 'CCI <= APO', 'DX <= 9MA', 'PrevHigh >= PrevAdjClose', 'PrevAdjClose >= STOCHSLOWK', 'ROC >= 20MA', 'DX <= PrevOpen', 'LowerBB >= Open', 'PrevVol >= STOCHSLOWK', 'BETA <= Open', 'APO <= TSF', 'ATR >= PrevHigh', 'STOCHSLOWD <= PrevAdjClose', 'MACDHIST <= RSI', 'OBV >= LINEARREG_SLOPE', 'CCI >= TSF', 'LINEARREG_SLOPE <= CCI', 'APO >= MACD', 'ULTOSC >= STOCHSLOWK', 'ROC <= LINEARREG', 'PrevOpen >= MidBB', 'DX <= ULTOSC', 'SAR >= OBV', 'LowerBB <= PrevOpen', 'OBV >= ROC', 'DX >= MACD', 'PrevVol <= UpperBB', 'STDDEV >= ROC', 'APO >= OBV', 'LINEARREG <= 20MA', 'MACDSIGNAL <= TSF', 'ADX <= 20MA', 'ADX >= APO', 'MACD >= 9MA', 'STDDEV <= STOCHSLOWK', 'MACDHIST <= ATR', 'MidBB >= ROC', 'LowerBB <= DX', 'MACD <= PrevVol', 'LINEARREG <= MidBB', 'PrevOpen >= LowerBB', 'STOCHSLOWK >= ULTOSC', 'ROC <= APO', 'MACDHIST <= PrevLow', 'OBV >= MOM', 'ROC >= LINEARREG_SLOPE', 'RSI >= LINEARREG', 'LINEARREG_ANGLE <= TSF', '9MA >= RSI', 'ADX >= LINEARREG_SLOPE', 'STDDEV <= MACDSIGNAL', 'ROC <= 9MA', 'PrevLow >= MidBB', 'MidBB >= ATR', 'TSF >= CCI', 'MACD <= STDDEV', 'ADX >= LINEARREG_ANGLE', 'Open >= TSF', 'PrevHigh <= CCI', 'MACDHIST <= BETA', 'PrevHigh >= MidBB', 'PrevOpen >= PrevLow', 'ULTOSC <= LINEARREG', '20MA >= MOM', 'MACDSIGNAL >= OBV', 'CCI <= PrevVol', 'MACDHIST <= LINEARREG', 'UpperBB >= TSF', 'MACD <= STOCHSLOWK', 'MOM <= STOCHSLOWD', 'STOCHSLOWK >= MACDHIST', 'STOCHSLOWK <= DX', 'LINEARREG_ANGLE >= Open', 'PrevOpen >= ATR', 'PrevOpen >= MOM', 'MACDSIGNAL >= MOM', 'MACDSIGNAL >= MidBB', 'LINEARREG_ANGLE <= RSI', 'STOCHSLOWD >= APO', 'MidBB >= STOCHSLOWK', 'LINEARREG <= LINEARREG_ANGLE', 'LINEARREG_ANGLE <= ULTOSC', 'MidBB >= Open', '9MA <= SAR', 'MidBB <= APO', 'MACD <= DX', 'PrevLow >= PrevOpen', 'LINEARREG_ANGLE >= PrevVol', 'CCI >= BETA', 'TSF >= ULTOSC', '9MA <= ATR', 'LowerBB >= LINEARREG_ANGLE', 'PrevAdjClose >= UpperBB', 'LINEARREG_ANGLE <= LINEARREG_SLOPE', '9MA >= STOCHSLOWK', 'LINEARREG >= PrevAdjClose', 'CCI <= OBV', 'CCI <= BETA', 'ATR <= LowerBB', 'PrevLow >= UpperBB', 'PrevHigh <= STOCHSLOWK', 'ATR <= LINEARREG', '20MA >= STOCHSLOWK', 'UpperBB <= MOM', 'UpperBB >= BETA', 'TSF <= PrevOpen', 'STOCHSLOWD <= STOCHSLOWK', '20MA <= PrevAdjClose', 'PrevAdjClose <= MACDSIGNAL', 'UpperBB <= PrevOpen', 'LINEARREG_SLOPE >= PrevOpen', 'SAR >= ADX', 'MACDSIGNAL <= STOCHSLOWK', 'LowerBB >= STDDEV', 'LowerBB <= 20MA', 'BETA >= ATR', 'MACD >= STDDEV', 'LINEARREG_ANGLE <= PrevVol', 'RSI >= 9MA', 'LINEARREG <= 9MA', 'LINEARREG <= CCI', 'UpperBB <= MACD', 'PrevLow >= APO', '20MA <= STOCHSLOWD', 'OBV >= APO', 'LINEARREG <= UpperBB', 'APO >= LINEARREG', '9MA >= CCI', 'RSI >= UpperBB', 'PrevVol <= CCI', '9MA <= STOCHSLOWK', 'ULTOSC <= MOM', '20MA >= ROC', 'BETA <= 20MA', 'TSF >= PrevOpen', 'TSF <= LINEARREG_SLOPE', 'MACDSIGNAL >= PrevAdjClose', 'PrevVol <= STDDEV', 'OBV >= TSF', 'LINEARREG_SLOPE >= DX', 'LINEARREG_ANGLE >= LINEARREG_SLOPE', 'PrevLow >= MOM', 'CCI >= SAR', 'PrevAdjClose >= LINEARREG_ANGLE', 'LINEARREG_SLOPE >= BETA', 'LINEARREG_SLOPE <= MACDSIGNAL', 'ROC >= MOM', 'STOCHSLOWD >= ROC', 'ATR >= LINEARREG', 'DX >= STOCHSLOWK', 'UpperBB >= APO', 'UpperBB <= OBV', 'ULTOSC >= LINEARREG_ANGLE', 'OBV >= ATR', 'UpperBB <= PrevVol', 'MACDHIST <= ROC', 'MOM <= RSI', 'TSF >= LowerBB', 'LINEARREG <= TSF', 'DX <= UpperBB', 'PrevOpen >= MACDHIST', 'MOM >= ULTOSC', 'UpperBB >= 9MA', 'ROC >= DX', 'ATR >= APO', 'PrevOpen >= MACDSIGNAL', 'LINEARREG_SLOPE <= ROC', 'MACDHIST >= MOM', 'STOCHSLOWD >= MOM', 'BETA <= PrevOpen', 'SAR >= Open', 'RSI <= CCI', 'DX <= PrevLow', 'SAR >= PrevHigh', 'Open >= CCI', 'CCI >= MACDSIGNAL', 'STOCHSLOWD >= SAR', 'STOCHSLOWK >= LowerBB', 'RSI <= PrevOpen', 'MOM <= ROC', '9MA <= STDDEV', 'MOM >= PrevLow', 'MidBB >= LINEARREG_SLOPE', 'BETA >= ROC', 'BETA >= PrevOpen', 'Open <= MACD', 'SAR >= UpperBB', 'MACDHIST <= PrevHigh', 'STDDEV <= TSF', 'MACD <= ULTOSC', 'OBV >= PrevAdjClose', 'PrevLow >= 20MA', 'STOCHSLOWD >= LINEARREG_SLOPE', 'PrevOpen <= CCI', 'PrevVol <= OBV', 'STOCHSLOWD >= Open', 'MACDSIGNAL >= PrevLow', 'MACDSIGNAL >= LINEARREG_SLOPE', 'PrevAdjClose >= MACD', 'LINEARREG_SLOPE <= Open', 'LINEARREG_SLOPE >= ROC', 'STOCHSLOWD <= MACDSIGNAL', 'LINEARREG_ANGLE >= MidBB', 'PrevHigh >= MOM', 'PrevOpen >= PrevAdjClose', 'BETA <= ULTOSC', 'TSF >= DX', 'MACDHIST <= 9MA', 'STDDEV <= DX', 'RSI <= LINEARREG', '20MA <= Open', 'ULTOSC <= MACD', 'Open >= LINEARREG_SLOPE', 'MACDHIST >= LowerBB', 'LINEARREG_SLOPE >= PrevHigh', 'PrevAdjClose >= STDDEV', 'LINEARREG_ANGLE >= ULTOSC', 'UpperBB <= ULTOSC', 'APO <= PrevLow', 'MOM >= STDDEV', 'OBV <= ULTOSC', 'MOM >= PrevOpen', 'STDDEV <= LINEARREG_ANGLE', 'MidBB <= LowerBB', 'PrevLow >= LINEARREG_ANGLE', 'BETA >= LINEARREG_SLOPE', 'STOCHSLOWD >= MACD', 'MidBB <= MACDHIST', 'ATR <= 9MA', 'UpperBB <= STOCHSLOWD', 'MACD >= 20MA', 'APO >= ADX', 'MidBB >= ADX', 'MACDHIST >= PrevOpen', 'ULTOSC >= MidBB', 'LINEARREG <= ROC', 'RSI <= PrevAdjClose', 'ADX >= PrevOpen', 'APO <= CCI', 'MOM <= ATR', 'MOM <= ADX', 'CCI >= PrevAdjClose', 'LowerBB >= DX', 'SAR <= UpperBB', 'PrevLow >= PrevAdjClose', 'PrevAdjClose <= PrevLow', 'STOCHSLOWD <= BETA', 'ATR >= MACD', 'STOCHSLOWD <= PrevHigh', 'TSF <= MACDHIST', 'CCI <= STDDEV', 'MACDSIGNAL >= DX', 'LowerBB <= ADX', 'PrevHigh >= Open', 'MACD >= PrevAdjClose', 'ADX >= MidBB', 'APO >= RSI', 'CCI >= RSI', 'PrevOpen >= Open', 'ROC >= UpperBB', 'PrevLow >= SAR', 'MACDSIGNAL <= DX', 'PrevHigh <= ROC', 'PrevLow >= LowerBB', 'PrevAdjClose >= ATR', 'TSF >= LINEARREG_ANGLE', 'ROC >= APO', 'PrevOpen <= OBV', 'PrevAdjClose <= PrevOpen', 'DX >= MOM', 'MACDSIGNAL <= PrevHigh', 'PrevLow >= STDDEV', 'CCI >= UpperBB', 'MACDHIST >= PrevVol', 'STOCHSLOWD >= UpperBB', '20MA >= ULTOSC', 'STOCHSLOWK >= ROC', '9MA >= ATR', 'MACD <= LINEARREG_SLOPE', 'PrevAdjClose >= CCI', 'LINEARREG_ANGLE >= DX', 'MACDHIST >= ADX', 'PrevLow <= UpperBB', 'LINEARREG_ANGLE >= MACDHIST', 'RSI <= Open', 'LINEARREG_ANGLE >= SAR', 'ADX <= PrevOpen', 'SAR >= BETA', 'MACDHIST <= ULTOSC', 'MidBB >= PrevVol', 'Open >= PrevHigh', 'PrevHigh <= LINEARREG_ANGLE', 'MACD <= MOM', 'PrevVol <= SAR', 'MACDHIST >= MidBB', 'MOM <= SAR', 'ULTOSC <= PrevHigh', 'MOM >= APO', 'LowerBB <= MOM', 'ULTOSC <= PrevLow', 'MACDHIST <= MOM', 'CCI >= LowerBB', 'ATR <= MACD', 'BETA <= ATR', 'PrevVol >= SAR', 'PrevVol <= DX', 'MOM <= LINEARREG', 'LowerBB <= OBV', 'LINEARREG <= PrevHigh', 'MACDHIST >= RSI', 'STOCHSLOWK >= PrevAdjClose', 'CCI >= APO', 'PrevOpen >= RSI', 'MACDSIGNAL <= STOCHSLOWD', 'MOM >= SAR', 'ROC >= PrevAdjClose', 'Open >= BETA', 'OBV <= PrevHigh', 'LINEARREG <= SAR', 'MACDSIGNAL <= LINEARREG', 'PrevVol <= ATR', 'APO <= DX', 'SAR <= STOCHSLOWK', '9MA >= PrevVol', 'BETA <= PrevVol', 'MACDHIST <= OBV', 'PrevOpen >= TSF', 'UpperBB >= PrevAdjClose', 'MACDSIGNAL <= PrevVol', 'DX >= MACDSIGNAL', 'STOCHSLOWD <= LowerBB', 'RSI >= ADX', 'PrevLow <= LINEARREG_ANGLE', 'LINEARREG_SLOPE >= ATR', 'UpperBB >= RSI', 'MOM >= 9MA', 'BETA >= STDDEV', 'ADX >= MACDHIST', 'RSI <= STDDEV', 'DX <= PrevVol', 'SAR <= PrevVol', 'CCI <= RSI', 'BETA >= SAR', 'ATR <= SAR', 'STDDEV >= UpperBB', '20MA <= ATR', 'ROC <= PrevOpen', 'TSF >= PrevVol', 'ADX >= OBV', 'UpperBB >= PrevOpen', 'STOCHSLOWK <= OBV', 'MACDSIGNAL >= CCI', 'STOCHSLOWD <= LINEARREG', 'CCI >= STDDEV', 'ROC >= 9MA', 'CCI >= STOCHSLOWD', 'OBV >= SAR', 'PrevAdjClose >= SAR', 'PrevAdjClose >= STOCHSLOWD', 'MACDSIGNAL <= LINEARREG_ANGLE', '20MA >= RSI', 'SAR >= ULTOSC', 'DX <= SAR', 'ULTOSC >= RSI', 'CCI <= LINEARREG', 'Open >= LINEARREG_ANGLE', 'MidBB <= STOCHSLOWK', 'LINEARREG_SLOPE >= ADX', 'RSI >= PrevOpen', 'MidBB', 'PrevOpen <= PrevVol', 'PrevAdjClose <= LowerBB', 'OBV >= MACDHIST', 'RSI >= STOCHSLOWD', 'RSI <= 9MA', 'MOM <= LINEARREG_SLOPE', 'STOCHSLOWK >= MACDSIGNAL', 'ADX <= MACD', 'LINEARREG <= STDDEV', 'MACDHIST <= DX', 'ULTOSC >= ROC', 'UpperBB >= ADX', 'RSI <= LINEARREG_ANGLE', 'PrevHigh >= LINEARREG', 'TSF <= PrevHigh', 'TSF >= ATR', 'PrevHigh >= BETA', 'MOM >= RSI', '20MA >= PrevHigh', 'APO <= LINEARREG', 'MACD <= ADX', 'LINEARREG <= MACDSIGNAL', '9MA <= BETA', 'TSF >= LINEARREG_SLOPE', 'PrevHigh >= UpperBB', 'DX <= MOM', '9MA >= ROC', 'MACDHIST >= STOCHSLOWK', 'APO >= PrevLow', '9MA <= APO', 'PrevOpen >= ULTOSC', 'ADX <= SAR', 'TSF >= STOCHSLOWK', 'STDDEV <= 9MA', 'UpperBB >= LINEARREG_ANGLE', 'ATR >= LINEARREG_ANGLE', 'LINEARREG >= MACD', 'LINEARREG >= TSF', 'TSF >= APO', '20MA <= 9MA', 'BETA >= MACDSIGNAL', 'PrevLow <= OBV', 'STOCHSLOWK <= LowerBB', 'PrevVol <= STOCHSLOWK', 'LowerBB <= APO', 'APO <= SAR', 'MOM <= TSF', 'APO >= MOM', 'DX >= LINEARREG_ANGLE', 'TSF <= MOM', 'STOCHSLOWK >= DX', 'SAR <= LINEARREG_SLOPE', 'LINEARREG >= OBV', 'MOM >= LINEARREG', 'STDDEV >= LINEARREG_SLOPE', 'PrevVol >= STOCHSLOWD', 'PrevAdjClose <= ROC', 'ROC >= Open', 'SAR', 'PrevLow <= PrevOpen', '9MA >= UpperBB', 'LINEARREG <= LowerBB', 'LINEARREG_SLOPE <= PrevAdjClose', 'MOM >= BETA', 'Open >= PrevAdjClose', 'LINEARREG', 'STOCHSLOWD >= MidBB', 'CCI >= PrevVol', 'ADX <= PrevLow', 'MOM >= TSF', 'STOCHSLOWD >= PrevLow', 'STOCHSLOWD <= DX', 'LowerBB >= APO', 'STOCHSLOWK <= STDDEV', 'SAR <= STOCHSLOWD', 'MACDHIST >= MACD', 'Open >= PrevOpen', '20MA <= STDDEV', 'Open >= MOM', 'MACDSIGNAL <= Open', 'ULTOSC >= DX', 'MACD >= ROC', 'ROC <= PrevVol', 'ADX <= OBV', 'PrevOpen >= LINEARREG_ANGLE', 'STDDEV >= MACD', 'MidBB >= TSF', 'RSI >= MACD', 'Open <= MACDHIST', 'LINEARREG >= ADX', 'ADX >= UpperBB', '20MA <= STOCHSLOWK', 'ROC >= MACDSIGNAL', 'MACDSIGNAL >= ROC', 'ADX >= MACDSIGNAL', 'PrevAdjClose >= BETA', 'LowerBB <= ATR', 'MidBB <= MOM', 'RSI <= PrevLow', 'MOM <= PrevAdjClose', 'PrevVol >= MACDSIGNAL', 'CCI >= MACD', 'Open >= PrevLow', 'PrevLow >= TSF', 'SAR >= RSI', 'ULTOSC <= PrevOpen', '20MA >= Open', 'ULTOSC <= OBV', '20MA >= CCI', 'APO >= LINEARREG_SLOPE', 'PrevVol >= DX', 'OBV <= STOCHSLOWD', 'SAR <= LINEARREG_ANGLE', 'MACDHIST >= DX', 'TSF <= BETA', 'LINEARREG_ANGLE >= APO', 'CCI <= STOCHSLOWK', 'APO >= MidBB', 'PrevHigh >= TSF', 'LINEARREG >= RSI', 'MOM >= MidBB', 'PrevVol <= PrevOpen', 'PrevVol >= PrevOpen', 'LINEARREG_SLOPE >= SAR', '9MA >= TSF', 'APO >= SAR', 'BETA >= STOCHSLOWD', 'STOCHSLOWD >= ADX', 'DX >= RSI', 'LINEARREG_ANGLE <= SAR', 'OBV <= Open', 'ADX <= PrevVol', 'OBV <= RSI', 'MACDSIGNAL >= 9MA', 'RSI <= MACDHIST', 'LINEARREG >= APO', 'ULTOSC >= 9MA', 'Open >= LowerBB', 'PrevLow <= ULTOSC', '9MA >= Open', '9MA >= ULTOSC', 'STOCHSLOWD >= ULTOSC', 'PrevOpen <= MACDHIST', 'LowerBB <= LINEARREG_SLOPE', 'CCI <= ULTOSC', 'LINEARREG <= MACDHIST', 'Open <= BETA', '9MA >= DX', 'Open >= APO', 'DX >= MACDHIST', 'MACD >= RSI', 'STDDEV <= RSI', 'PrevOpen >= UpperBB', 'MidBB <= PrevAdjClose', 'MACDSIGNAL <= 20MA', 'ATR <= DX', 'STOCHSLOWD <= MOM', 'APO <= RSI', 'LINEARREG <= LINEARREG_SLOPE', 'OBV >= MACDSIGNAL', 'LINEARREG_ANGLE >= RSI', 'RSI >= OBV', 'PrevLow >= RSI', 'PrevOpen <= APO', 'CCI <= MOM', 'SAR >= MOM', 'TSF >= RSI', 'ATR <= PrevOpen', 'SAR <= MACD', 'MidBB >= MACDSIGNAL', 'STOCHSLOWK <= 9MA', 'STOCHSLOWD >= STDDEV', 'ADX >= PrevAdjClose', 'ULTOSC >= PrevOpen', 'LowerBB >= PrevHigh', 'MACDSIGNAL >= MACD', 'UpperBB <= RSI', 'APO <= STOCHSLOWD', 'LINEARREG_SLOPE <= OBV', 'MOM <= DX', 'STOCHSLOWK >= PrevOpen', 'TSF >= MACDSIGNAL', 'PrevLow <= BETA', 'ADX <= CCI', 'Open >= DX', 'LowerBB >= STOCHSLOWK', 'LowerBB >= LINEARREG', 'MOM >= LINEARREG_ANGLE', 'LowerBB <= MACD', 'ROC >= OBV', 'LINEARREG_SLOPE <= TSF', 'PrevVol <= BETA', 'LINEARREG_SLOPE >= 9MA', 'PrevHigh >= APO', 'PrevAdjClose <= STOCHSLOWK', 'MACD <= PrevHigh', 'APO <= PrevVol', 'BETA >= MidBB', 'PrevLow <= RSI', 'DX >= OBV', 'RSI <= OBV', '20MA <= MidBB', 'RSI >= 20MA', 'ADX <= ATR', 'PrevAdjClose <= PrevHigh', 'MACDHIST >= LINEARREG', 'SAR >= STDDEV', 'STOCHSLOWD <= Open', 'OBV >= CCI', '20MA <= PrevVol', 'LINEARREG_SLOPE >= MACD', 'ATR >= STOCHSLOWK', 'TSF <= SAR', 'ROC >= LowerBB', 'RSI >= MACDHIST', 'LINEARREG_ANGLE <= 20MA', 'Open >= OBV', 'APO <= 9MA', 'MOM >= CCI', 'ATR <= BETA', 'MOM <= MACDSIGNAL', 'RSI >= ULTOSC', 'PrevOpen >= LINEARREG', 'LowerBB >= BETA', 'ROC <= MACDSIGNAL', 'STDDEV <= Open', 'PrevOpen <= ULTOSC', 'CCI <= ADX', 'PrevVol <= LowerBB', 'STDDEV >= PrevAdjClose', 'MOM <= PrevHigh', 'STOCHSLOWD >= CCI', 'LINEARREG_SLOPE >= MACDSIGNAL', 'LINEARREG >= ATR', 'Open <= STOCHSLOWK', 'UpperBB <= APO', 'ATR <= ROC', 'UpperBB >= MACDSIGNAL', 'TSF <= PrevLow', 'MACDHIST >= SAR', 'APO >= DX', 'ADX <= LINEARREG', 'ROC <= OBV', 'STOCHSLOWD <= TSF', 'STOCHSLOWK >= APO', 'SAR >= TSF', 'ADX >= STOCHSLOWD', 'MACDHIST >= PrevHigh', 'BETA >= Open', 'APO >= STDDEV', 'PrevLow <= MACD', 'OBV >= RSI', '20MA <= RSI', 'LowerBB >= ULTOSC', 'CCI <= ATR', 'ATR >= BETA', 'MACD >= PrevOpen', 'PrevAdjClose <= ADX', 'STOCHSLOWK >= BETA', 'LowerBB <= UpperBB', 'OBV >= DX', 'ATR <= Open', 'LINEARREG_ANGLE >= LowerBB', 'LowerBB >= TSF', 'ATR <= 20MA', 'ROC >= MidBB', 'RSI <= DX', 'PrevHigh >= LowerBB', 'STDDEV >= 9MA', 'PrevAdjClose >= TSF', 'STDDEV >= LowerBB', 'STDDEV <= 20MA', 'BETA >= PrevHigh', 'MidBB <= OBV', 'ATR >= CCI', 'ATR <= PrevLow', 'MACDSIGNAL <= UpperBB', 'STDDEV >= PrevVol', 'LowerBB >= PrevAdjClose', '20MA >= DX', 'TSF <= STDDEV', 'PrevVol >= TSF', 'MACD >= BETA', 'ULTOSC >= PrevAdjClose', 'SAR >= PrevVol', 'RSI >= CCI', 'UpperBB >= MOM', 'PrevVol >= 9MA', 'UpperBB >= DX', 'ROC >= ATR', 'BETA >= UpperBB', 'SAR >= 9MA', 'MACDHIST >= STOCHSLOWD', 'ULTOSC <= LowerBB', 'TSF <= LINEARREG', 'BETA <= UpperBB', 'MidBB >= PrevLow', 'MACD <= STOCHSLOWD', 'PrevHigh >= ROC', 'MidBB <= ADX', 'PrevHigh >= CCI', 'ATR >= ROC', 'LINEARREG >= LowerBB', 'Open >= ATR', 'CCI >= LINEARREG', 'ATR <= MACDHIST', 'APO <= MACD', 'ULTOSC >= PrevLow', 'PrevOpen <= BETA', 'STOCHSLOWK <= PrevVol', 'PrevVol >= 20MA', 'MACD <= Open', 'STDDEV <= LowerBB', 'RSI >= LowerBB', 'LowerBB >= 9MA', 'STDDEV >= Open', 'MidBB >= MACDHIST', 'LINEARREG >= BETA', 'BETA <= 9MA', 'STOCHSLOWK >= MOM', 'STOCHSLOWK <= PrevLow', 'PrevHigh >= ATR', 'ULTOSC >= STOCHSLOWD', 'STOCHSLOWD <= MidBB', 'STOCHSLOWK >= SAR', '20MA >= UpperBB', 'DX >= 20MA', 'LINEARREG_SLOPE <= SAR', 'LINEARREG >= MACDHIST', 'PrevOpen >= LINEARREG_SLOPE', 'MACD >= LINEARREG', 'MACDSIGNAL <= APO', 'CCI >= ATR', 'LINEARREG_SLOPE <= ULTOSC', 'OBV <= 20MA', 'LINEARREG_ANGLE <= UpperBB', 'ROC <= PrevHigh', 'ROC <= LINEARREG_SLOPE', 'ROC >= ADX', 'PrevLow >= MACDSIGNAL', 'PrevAdjClose >= MidBB', 'ADX >= CCI', 'UpperBB <= Open', '9MA <= UpperBB', 'PrevVol >= CCI', 'ULTOSC >= UpperBB', 'MACDHIST <= MACD', 'CCI >= DX', 'OBV <= MACDSIGNAL', 'STDDEV >= CCI', 'MACDHIST <= STDDEV', 'LowerBB <= MACDHIST', 'MACD <= MACDSIGNAL', 'ULTOSC <= LINEARREG_ANGLE', 'RSI >= Open', 'MidBB <= PrevLow', '9MA <= ULTOSC', 'ATR >= ULTOSC', 'LINEARREG >= PrevLow', 'LINEARREG_SLOPE <= UpperBB', 'ULTOSC >= LINEARREG', 'PrevVol <= ROC', 'ATR >= LowerBB', 'PrevOpen <= ADX', 'BETA <= TSF', 'OBV <= APO', '20MA <= UpperBB', '9MA <= RSI', 'APO <= STDDEV', 'APO <= PrevHigh', 'STOCHSLOWD <= ATR', 'ROC <= STOCHSLOWD', 'RSI <= APO', 'ULTOSC <= TSF', 'MidBB >= ULTOSC', 'MACDHIST <= ADX', 'PrevHigh <= ULTOSC', 'UpperBB >= LowerBB', 'Open >= MACD', 'OBV <= LowerBB', 'STDDEV >= BETA', 'CCI >= PrevOpen', 'TSF >= SAR', 'ULTOSC <= LINEARREG_SLOPE', 'RSI >= STDDEV', 'MidBB >= CCI', 'MidBB <= DX', 'UpperBB >= OBV', '9MA <= 20MA', 'UpperBB <= MidBB', 'PrevAdjClose >= LINEARREG_SLOPE', 'MACD <= LINEARREG_ANGLE', 'BETA >= PrevLow', 'STOCHSLOWK >= CCI', 'ADX >= DX', 'PrevVol >= BETA', 'PrevVol >= ULTOSC', 'TSF <= STOCHSLOWD', 'APO >= ULTOSC', 'RSI <= UpperBB', 'PrevHigh <= MACDHIST', 'MACD >= LowerBB', 'MOM >= MACDSIGNAL', '20MA >= MACDSIGNAL', 'RSI <= SAR', 'ADX >= 20MA', 'DX >= STDDEV', 'ROC >= MACDHIST', 'ROC >= STOCHSLOWK', 'BETA <= RSI', 'OBV >= BETA', 'ADX <= LINEARREG_SLOPE', 'MidBB >= LINEARREG', 'OBV <= LINEARREG_SLOPE', 'BETA <= MACDSIGNAL', 'STOCHSLOWK >= 9MA', 'TSF <= ULTOSC', 'ATR >= MOM', 'Open <= LINEARREG', 'DX >= LINEARREG', 'ADX <= MOM', 'CCI <= MACDHIST', 'LINEARREG_SLOPE <= PrevLow', 'LINEARREG_ANGLE >= STDDEV', '9MA <= CCI', 'STDDEV >= PrevLow', '20MA <= ROC', 'PrevOpen <= ROC', 'PrevOpen <= MACD', 'BETA >= DX', 'MidBB <= LINEARREG_SLOPE', 'DX <= STOCHSLOWK', 'LINEARREG >= MOM', 'SAR >= MidBB', 'PrevAdjClose >= RSI', 'ADX <= PrevAdjClose', 'LINEARREG_SLOPE <= STOCHSLOWK', 'STDDEV >= 20MA', 'STOCHSLOWK >= UpperBB', 'LowerBB <= STOCHSLOWD', 'PrevOpen <= STOCHSLOWD', 'PrevHigh >= STDDEV', 'SAR <= LINEARREG', 'DX <= BETA', 'CCI >= PrevLow', 'CCI >= Open', 'LINEARREG_ANGLE <= MACD', 'DX <= ADX', 'PrevLow <= CCI', 'BETA >= LINEARREG_ANGLE', 'STOCHSLOWK <= RSI', '20MA <= APO', 'STOCHSLOWD <= 9MA', 'OBV >= 9MA', 'MACDSIGNAL <= ULTOSC', 'PrevHigh >= PrevOpen', 'RSI <= PrevHigh', 'LINEARREG_SLOPE <= PrevVol', 'MACD <= SAR', 'PrevOpen <= LINEARREG_SLOPE', 'ADX >= SAR', 'SAR <= ROC', 'MOM >= ADX', 'Open >= ADX', 'APO >= UpperBB', 'DX >= APO', 'LowerBB <= PrevAdjClose', 'PrevVol >= STDDEV', 'ROC >= RSI', 'ROC >= PrevLow', 'PrevHigh <= 9MA', 'LINEARREG_ANGLE <= BETA', 'CCI <= 20MA', 'DX >= CCI', 'Open >= ROC', 'LINEARREG >= STDDEV', 'APO >= Open', 'PrevHigh >= ULTOSC', '20MA >= SAR', 'LowerBB >= ADX', 'STDDEV <= PrevLow', 'PrevVol >= UpperBB', 'ULTOSC <= 9MA', 'LINEARREG >= STOCHSLOWD', 'PrevVol <= PrevLow', 'ULTOSC >= PrevVol', 'PrevVol <= 20MA', 'RSI <= STOCHSLOWD', '20MA <= MOM', 'PrevVol <= MACDHIST', '20MA >= 9MA', 'PrevLow <= MidBB', 'CCI >= OBV', 'SAR <= PrevHigh', 'LINEARREG_SLOPE >= UpperBB', 'LINEARREG_SLOPE <= MACD', 'APO <= PrevOpen', 'PrevAdjClose >= ROC', 'TSF <= MidBB', 'MACDSIGNAL >= LINEARREG', 'STDDEV >= ADX', 'ROC <= SAR', 'ATR <= OBV', 'LowerBB <= TSF', 'MACDSIGNAL >= LowerBB', 'MOM <= CCI', '9MA >= LowerBB', 'SAR >= CCI', 'ADX <= Open', 'MACDSIGNAL >= STDDEV', 'TSF >= BETA', 'UpperBB <= ATR', 'PrevVol <= MOM', 'MOM <= MidBB', 'MACDSIGNAL >= PrevVol', 'ADX <= DX', '9MA <= PrevOpen', 'STOCHSLOWK <= MACDSIGNAL', 'PrevVol <= 9MA', 'LINEARREG_ANGLE <= ROC', 'ROC <= ATR', 'UpperBB >= STOCHSLOWK', 'MACD <= 20MA', 'PrevVol >= ROC', 'RSI >= MACDSIGNAL', 'MACDSIGNAL <= ADX', 'ATR <= LINEARREG_ANGLE', 'MOM >= LowerBB', '20MA <= TSF', 'TSF <= STOCHSLOWK', 'LINEARREG_SLOPE <= RSI', 'MOM <= BETA', 'DX <= PrevHigh', 'APO <= ADX', 'APO >= ATR', '9MA >= PrevHigh', 'LINEARREG_SLOPE <= PrevOpen', '9MA >= STDDEV', 'UpperBB <= DX', 'MACD >= SAR', 'SAR >= STOCHSLOWK', 'PrevHigh <= LINEARREG_SLOPE', 'CCI >= MOM', 'MidBB <= RSI', 'PrevVol >= PrevAdjClose', 'LINEARREG_SLOPE >= Open', 'ROC <= PrevLow', 'STDDEV >= PrevOpen', 'TSF <= 20MA', 'MACD >= ULTOSC', 'PrevLow >= LINEARREG', 'Open <= LINEARREG_SLOPE', 'MOM <= LINEARREG_ANGLE', 'MACDSIGNAL <= ROC', 'LINEARREG_SLOPE >= 20MA', 'MACDHIST >= PrevAdjClose', '9MA >= PrevAdjClose', 'SAR >= DX', 'LINEARREG <= PrevVol', 'UpperBB <= CCI', 'STOCHSLOWD >= PrevAdjClose', 'LowerBB <= MACDSIGNAL', 'BETA <= ADX', 'STDDEV <= PrevVol', 'LowerBB >= ROC', 'ROC >= STOCHSLOWD', 'APO <= MidBB', 'MACDHIST <= LowerBB', 'MACD <= OBV', 'ADX <= ULTOSC', 'MACD >= APO', 'ADX <= MACDSIGNAL', 'RSI <= ATR', 'BETA >= ULTOSC', 'TSF', 'BETA <= PrevHigh', 'PrevLow >= DX', 'STOCHSLOWK <= CCI', 'STOCHSLOWK <= LINEARREG_ANGLE', '20MA >= LowerBB', 'STOCHSLOWD <= PrevVol', '9MA <= STOCHSLOWD', 'OBV >= ULTOSC', 'APO <= ATR', 'PrevLow <= PrevAdjClose', 'PrevHigh <= MACD', 'MidBB <= STOCHSLOWD', 'STDDEV >= ATR', 'TSF <= OBV', 'STOCHSLOWK >= Open', 'PrevHigh <= MACDSIGNAL', 'LINEARREG >= LINEARREG_SLOPE', 'MACD <= PrevAdjClose', 'MACDSIGNAL <= PrevOpen', 'RSI >= PrevAdjClose', 'TSF >= PrevAdjClose', 'MACDHIST <= PrevVol', 'PrevVol <= Open', 'ATR >= RSI', '9MA <= LowerBB', '20MA >= APO', 'ULTOSC >= LowerBB', 'Open <= ULTOSC', 'LINEARREG_ANGLE >= CCI', 'BETA <= MACD', 'APO >= 20MA', 'MACD <= UpperBB', 'RSI >= PrevLow', '9MA <= ROC', 'MACD >= STOCHSLOWD', 'PrevVol <= MidBB', 'TSF >= OBV', 'MOM <= LowerBB', 'PrevAdjClose >= PrevOpen', 'SAR <= PrevAdjClose', 'MACD >= LINEARREG_SLOPE', 'UpperBB >= PrevVol', 'TSF <= PrevVol', 'PrevHigh <= ADX', 'STOCHSLOWD <= CCI', 'RSI <= MACD', 'CCI >= LINEARREG_ANGLE', 'CCI <= LINEARREG_ANGLE', 'APO <= PrevAdjClose', 'ATR >= MACDSIGNAL', 'MACDHIST >= UpperBB', 'PrevAdjClose <= LINEARREG', 'ULTOSC >= LINEARREG_SLOPE', 'APO <= ULTOSC', '9MA >= SAR', 'ROC <= 20MA', 'ROC >= LINEARREG_ANGLE', 'PrevLow <= APO', 'STOCHSLOWD <= 20MA', 'DX >= TSF', 'ADX <= TSF', 'DX >= ROC', 'OBV <= PrevOpen', 'LowerBB >= PrevVol', 'LINEARREG_ANGLE <= MACDSIGNAL', 'PrevOpen <= LowerBB', 'LINEARREG_SLOPE <= 9MA', 'LINEARREG_SLOPE >= ULTOSC', 'STOCHSLOWD <= OBV', 'PrevAdjClose <= RSI', 'PrevHigh >= MACDSIGNAL', 'ROC <= LowerBB', 'PrevAdjClose <= DX', 'STOCHSLOWD >= RSI', 'PrevHigh <= ATR', 'ATR >= TSF', 'RSI >= LINEARREG_SLOPE', 'OBV <= ADX', 'APO >= TSF', 'LINEARREG_ANGLE <= Open', 'BETA <= PrevAdjClose']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['missing', 'single_unique', 'collinear', 'zero_importance', 'low_importance'] methods have been run\n",
      "\n",
      "Removed 2034 features.\n",
      "            9MA   ADX  APO    CCI    DX  MACD  MACDSIGNAL  MACDHIST   MOM  \\\n",
      "Date                                                                        \n",
      "2002-07-12 1.16 23.49 0.04  91.31 28.10  0.02       -0.03      0.05  0.28   \n",
      "2002-07-15 1.19 23.84 0.06  82.68 28.28  0.02       -0.02      0.04  0.26   \n",
      "2002-07-16 1.20 22.32 0.07  27.82  2.62  0.02       -0.01      0.03  0.14   \n",
      "2002-07-17 1.20 20.91 0.08   1.23  2.62  0.02       -0.01      0.02  0.10   \n",
      "2002-07-18 1.19 19.59 0.09 -28.09  2.34  0.01       -0.00      0.01 -0.08   \n",
      "\n",
      "             ROC     ...       UpperBB <= PrevAdjClose  Open <= PrevHigh  \\\n",
      "Date                 ...                                                   \n",
      "2002-07-12 28.40     ...                         False              True   \n",
      "2002-07-15 28.01     ...                         False              True   \n",
      "2002-07-16 14.37     ...                         False              True   \n",
      "2002-07-17  9.59     ...                         False              True   \n",
      "2002-07-18 -7.13     ...                         False              True   \n",
      "\n",
      "            PrevOpen  PrevHigh  PrevLow  PrevAdjClose      PrevVol  Open  \\\n",
      "Date                                                                       \n",
      "2002-07-12      1.15      1.28     1.13          1.27 2,529,800.00  1.28   \n",
      "2002-07-15      1.28      1.28     1.21          1.21 2,837,800.00  1.24   \n",
      "2002-07-16      1.24      1.25     1.08          1.14 4,239,200.00  1.15   \n",
      "2002-07-17      1.15      1.16     1.10          1.13 1,941,800.00  1.14   \n",
      "2002-07-18      1.14      1.16     1.08          1.09 1,261,400.00  1.12   \n",
      "\n",
      "            Adj Close  DailyResult  \n",
      "Date                                \n",
      "2002-07-12       1.21            0  \n",
      "2002-07-15       1.14            0  \n",
      "2002-07-16       1.13            0  \n",
      "2002-07-17       1.09            0  \n",
      "2002-07-18       1.15            1  \n",
      "\n",
      "[5 rows x 114 columns]\n",
      "Our wonderful, new dataset has been created!\n"
     ]
    }
   ],
   "source": [
    "# Let's go!\n",
    "#df = df.set_index('Date')\n",
    "\n",
    "def feature_removal_func(df):\n",
    "    \n",
    "# Name all the feature columns and target label column for FeatureSelector...\n",
    "# Save the open price as FS removes it due to high correlation, but we want to keep it    \n",
    "    openprice = df[\"Open\"]\n",
    "    features = df.drop([\"Adj Close\",\"DailyResult\",'PrevOpen','PrevHigh','PrevLow','PrevAdjClose','PrevVol'], axis=1)\n",
    "    label = df[\"DailyResult\"]\n",
    "    adjclose = df[\"Adj Close\"]\n",
    "    preop = df['PrevOpen']\n",
    "    prehi = df['PrevHigh']\n",
    "    prelo = df['PrevLow']\n",
    "    preadjclose = df['PrevAdjClose']\n",
    "    prevol = df['PrevVol']\n",
    "\n",
    "# Now, drop all columns of low importance, crazy high correlation, etc.\n",
    "    from feature_selector import FeatureSelector\n",
    "    fs = FeatureSelector(data = features, labels = label)\n",
    "    fs.identify_all(selection_params = {'missing_threshold': 0.6,    \n",
    "                                    'correlation_threshold': 0.98, \n",
    "                                    'task': 'classification',    \n",
    "                                    'eval_metric': 'auc', \n",
    "                                    'cumulative_importance': 0.99})\n",
    "    all_to_remove = fs.check_removal()\n",
    "    print(all_to_remove[:])\n",
    "    df = fs.remove(methods = 'all')\n",
    "\n",
    "# Re-Add the Open. And Adj Close to the df because FeatureTools removes it once you assign it as the label for some reason\n",
    "    df['PrevOpen'] = preop\n",
    "    df['PrevHigh'] = prehi\n",
    "    df['PrevLow'] = prelo\n",
    "    df['PrevAdjClose'] = preadjclose\n",
    "    df['PrevVol'] = prevol\n",
    "    df['Open'] = openprice\n",
    "    df[\"Adj Close\"] = adjclose\n",
    "    df['DailyResult'] = label\n",
    "    return(df)\n",
    "\n",
    "df = feature_removal_func(df)\n",
    "\n",
    "print(df.head())\n",
    "print('Our wonderful, new dataset has been created!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TA-Lib also has candlestick patterns available too, so let's add those in.\n",
    "# We add them in after because their values don't (necessarily) need to be considered\n",
    "# in the feature selection part. We either see a candlestick pattern or we don't.\n",
    "# There's a lot of them:\n",
    "# CDL2CROWS - Two Crows\n",
    "#integer = talib.CDL2CROWS(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDL2CROWS'] = integer\n",
    "# CDL3BLACKCROWS - Three Black Crows\n",
    "#integer = talib.CDL3BLACKCROWS(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDL3BLACKCROWS'] = integer\n",
    "# CDL3INSIDE - Three Inside Up/Down\n",
    "#integer = talib.CDL3INSIDE(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDL3INSIDE'] = #integer\n",
    "# CDL3LINESTRIKE - T#hree-Line Strike\n",
    "#integer = talib.CDL3LINESTRIKE(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDL3LI#NESTRIKE'#] = integer\n",
    "# CDL3OUTSI#DE - Thre#e Outside Up/Down\n",
    "#integer = talib.CDL3OUTSIDE(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDL3OU#TSIDE'] =# integer\n",
    "# CDL3STARS#INSOUTH -# Three Stars In The South\n",
    "#integer = talib.CDL3STARSINSOUTH(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDL3ST#ARSINSOUTH'] = integer\n",
    "# CDL3WHITE#SOLDIERS - Three Advancing White Soldiers\n",
    "#integer = talib.CDL3WHITESOLDIERS(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDL3WH#ITESOLDIERS'] = integer\n",
    "# CDLABANDO#NEDBABY - Abandoned Baby\n",
    "#integer = talib.CDLABANDONEDBABY(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'], penetration=0)\n",
    "#df['CDLABA#NDONEDBABY'] = integer\n",
    "# CDLADVANC#EBLOCK - Advance Block\n",
    "#integer = talib.CDLADVANCEBLOCK(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLADV#ANCEBLOCK'] = integer\n",
    "# CDLBELTHO#LD - Belt-hold\n",
    "#integer = talib.CDLBELTHOLD(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLBEL#THOLD'] = integer\n",
    "# CDLBREAKAWAY - Breakaway\n",
    "#integer = talib.CDLBREAKAWAY(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLBREAKAWAY'] = integer\n",
    "# CDLCLOSINGMARUBOZU - Closing Marubozu\n",
    "#integer = talib.CDLCLOSINGMARUBOZU(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLCLOSINGMARUBOZU'] = integer\n",
    "# CDLCONCEALBABYSWALL - Concealing Baby Swallow\n",
    "#integer = talib.CDLCONCEALBABYSWALL(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLCONCEALBABYSWALL'] = integer\n",
    "# CDLCOUNTERATTACK - Counterattack\n",
    "#integer = talib.CDLCOUNTERATTACK(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLCOUNTERATTACK'] = integer\n",
    "# CDLDARKCLOUDCOVER - Dark Cloud Cover\n",
    "#integer = talib.CDLDARKCLOUDCOVER(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'], penetration=0)\n",
    "#df['CDLDARKCLOUDCOVER'] = integer\n",
    "# CDLDOJI - Doji\n",
    "integer = talib.CDLDOJI(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLDOJI'] = integer\n",
    "# CDLDOJISTAR - Doji Star\n",
    "integer = talib.CDLDOJISTAR(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLDOJISTAR'] = integer\n",
    "# CDLDRAGONFLYDOJI - Dragonfly Doji\n",
    "integer = talib.CDLDRAGONFLYDOJI(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLDRAGONFLYDOJI'] = integer\n",
    "# CDLENGULFING - Engulfing Pattern\n",
    "#integer = talib.CDLENGULFING(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLENGULFING'] = integer\n",
    "# CDLEVENINGDOJISTAR - Evening Doji Star\n",
    "integer = talib.CDLEVENINGDOJISTAR(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'], penetration=0)\n",
    "df['CDLEVENINGDOJISTAR'] = integer\n",
    "# CDLEVENINGSTAR - Evening Star\n",
    "integer = talib.CDLEVENINGSTAR(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'], penetration=0)\n",
    "df['CDLEVENINGSTAR'] = integer\n",
    "# CDLGAPSIDESIDEWHITE - Up/Down-gap side-by-side white lines\n",
    "#integer = talib.CDLGAPSIDESIDEWHITE(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLGAPSIDESIDEWHITE'] = integer\n",
    "# CDLGRAVESTONEDOJI - Gravestone Doji\n",
    "integer = talib.CDLGRAVESTONEDOJI(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLGRAVESTONEDOJI'] = integer\n",
    "# CDLHAMMER - Hammer\n",
    "integer = talib.CDLHAMMER(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLHAMMER'] = integer\n",
    "# CDLHANGINGMAN - Hanging Man\n",
    "#integer = talib.CDLHANGINGMAN(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLHANGINGMAN'] = integer\n",
    "# CDLHARAMI - Harami Pattern\n",
    "integer = talib.CDLHARAMI(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLHARAMI'] = integer\n",
    "# CDLHARAMICROSS - Harami Cross Pattern\n",
    "integer = talib.CDLHARAMICROSS(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLHARAMICROSS'] = integer\n",
    "# CDLHIGHWAVE - High-Wave Candle\n",
    "#integer = talib.CDLHIGHWAVE(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLHIGHWAVE'] = integer\n",
    "# CDLHIKKAKE - Hikkake Pattern\n",
    "integer = talib.CDLHIKKAKE(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLHIKKAKE'] = integer\n",
    "# CDLHIKKAKEMOD - Modified Hikkake Pattern\n",
    "#integer = talib.CDLHIKKAKEMOD(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLHIKKAKEMOD'] = integer\n",
    "# CDLHOMINGPIGEON - Homing Pigeon\n",
    "#integer = talib.CDLHOMINGPIGEON(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLHOMINGPIGEON'] = integer\n",
    "# CDLIDENTICAL3CROWS - Identical Three Crows\n",
    "#integer = talib.CDLIDENTICAL3CROWS(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLIDENTICAL3CROWS'] = integer\n",
    "# CDLINNECK - In-Neck Pattern\n",
    "#integer = talib.CDLINNECK(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLINNECK'] = integer\n",
    "# CDLINVERTEDHAMMER - Inverted Hammer\n",
    "integer = talib.CDLINVERTEDHAMMER(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLINVERTEDHAMMER'] = integer\n",
    "# CDLKICKING - Kicking\n",
    "#integer = talib.CDLKICKING(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLKICKING'] = integer\n",
    "# CDLKICKINGBYLENGTH - Kicking - bull/bear determined by the longer marubozu\n",
    "#integer = talib.CDLKICKINGBYLENGTH(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLKICKINGBYLENGTH'] = integer\n",
    "# CDLLADDERBOTTOM - Ladder Bottom\n",
    "#integer = talib.CDLLADDERBOTTOM(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLLADDERBOTTOM'] = integer\n",
    "# CDLLONGLEGGEDDOJI - Long Legged Doji\n",
    "integer = talib.CDLLONGLEGGEDDOJI(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLLONGLEGGEDDOJI'] = integer\n",
    "# CDLLONGLINE - Long Line Candle\n",
    "#integer = talib.CDLLONGLINE(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLLONGLINE'] = integer\n",
    "# CDLMARUBOZU - Marubozu\n",
    "#integer = talib.CDLMARUBOZU(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLMARUBOZU'] = integer\n",
    "# CDLMATCHINGLOW - Matching Low\n",
    "#integer = talib.CDLMATCHINGLOW(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLMATCHINGLOW'] = integer\n",
    "# CDLMATHOLD - Mat Hold\n",
    "#integer = talib.CDLMATHOLD(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'], penetration=0)\n",
    "#df['CDLMATHOLD'] = integer\n",
    "# CDLMORNINGDOJISTAR - Morning Doji Star\n",
    "integer = talib.CDLMORNINGDOJISTAR(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'], penetration=0)\n",
    "df['CDLMORNINGDOJISTAR'] = integer\n",
    "# CDLMORNINGSTAR - Morning Star\n",
    "integer = talib.CDLMORNINGSTAR(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'], penetration=0)\n",
    "df['CDLMORNINGSTAR'] = integer\n",
    "# CDLONNECK - On-Neck Pattern\n",
    "#integer = talib.CDLONNECK(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLONNECK'] = integer\n",
    "# CDLPIERCING - Piercing Pattern\n",
    "#integer = talib.CDLPIERCING(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLPIERCING'] = integer\n",
    "# CDLRICKSHAWMAN - Rickshaw Man\n",
    "#integer = talib.CDLRICKSHAWMAN(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLRICKSHAWMAN'] = integer\n",
    "# CDLRISEFALL3METHODS - Rising/Falling Three Methods\n",
    "#integer = talib.CDLRISEFALL3METHODS(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLRISEFALL3METHODS'] = integer\n",
    "# CDLSEPARATINGLINES - Separating Lines\n",
    "#integer = talib.CDLSEPARATINGLINES(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLSEPARATINGLINES'] = integer\n",
    "# CDLSHOOTINGSTAR - Shooting Star\n",
    "integer = talib.CDLSHOOTINGSTAR(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLSHOOTINGSTAR'] = integer\n",
    "# CDLSHORTLINE - Short Line Candle\n",
    "#integer = talib.CDLSHORTLINE(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLSHORTLINE'] = integer\n",
    "# CDLSPINNINGTOP - Spinning Top\n",
    "integer = talib.CDLSPINNINGTOP(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLSPINNINGTOP'] = integer\n",
    "# CDLSTALLEDPATTERN - Stalled Pattern\n",
    "#integer = talib.CDLSTALLEDPATTERN(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLSTALLEDPATTERN'] = integer\n",
    "# CDLSTICKSANDWICH - Stick Sandwich\n",
    "#integer = talib.CDLSTICKSANDWICH(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLSTICKSANDWICH'] = integer\n",
    "# CDLTAKURI - Takuri (Dragonfly Doji with very long lower shadow)\n",
    "integer = talib.CDLTAKURI(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "df['CDLTAKURI'] = integer\n",
    "# CDLTASUKIGAP - Tasuki Gap\n",
    "#integer = talib.CDLTASUKIGAP(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLTASUKIGAP'] = integer\n",
    "# CDLTHRUSTING - Thrusting Pattern\n",
    "#integer = talib.CDLTHRUSTING(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLTHRUSTING'] = integer\n",
    "# CDLTRISTAR - Tristar Pattern\n",
    "#integer = talib.CDLTRISTAR(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLTRISTAR'] = integer\n",
    "# CDLUNIQUE3RIVER - Unique 3 River\n",
    "#integer = talib.CDLUNIQUE3RIVER(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLUNIQUE3RIVER'] = integer\n",
    "# CDLUPSIDEGAP2CROWS - Upside Gap Two Crows\n",
    "#integer = talib.CDLUPSIDEGAP2CROWS(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLUPSIDEGAP2CROWS'] = integer\n",
    "# CDLXSIDEGAP3METHODS - Upside/Downside Gap Three Methods\n",
    "#integer = talib.CDLXSIDEGAP3METHODS(df['PrevOpen'], df['PrevHigh'], df['PrevLow'], df['PrevAdjClose'])\n",
    "#df['CDLXSIDEGAP3METHODS'] = integer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              9MA   ADX    APO     CCI    DX   MACD  MACDSIGNAL  MACDHIST  \\\n",
      "Date                                                                        \n",
      "2002-07-12   1.16 23.49   0.04   91.31 28.10   0.02       -0.03      0.05   \n",
      "2002-07-15   1.19 23.84   0.06   82.68 28.28   0.02       -0.02      0.04   \n",
      "2002-07-16   1.20 22.32   0.07   27.82  2.62   0.02       -0.01      0.03   \n",
      "2002-07-17   1.20 20.91   0.08    1.23  2.62   0.02       -0.01      0.02   \n",
      "2002-07-18   1.19 19.59   0.09  -28.09  2.34   0.01       -0.00      0.01   \n",
      "2002-07-19   1.18 18.32   0.10  -10.21  1.81   0.01        0.00      0.01   \n",
      "2002-07-22   1.17 17.14   0.10  -21.86  1.81   0.01        0.00      0.01   \n",
      "2002-07-23   1.17 16.06   0.09  -29.98  2.01   0.01        0.00      0.01   \n",
      "2002-07-24   1.17 15.54   0.08  -18.32  8.82   0.01        0.01      0.01   \n",
      "2002-07-25   1.15 14.65   0.06  -98.47  3.11   0.01        0.01      0.00   \n",
      "2002-07-26   1.13 14.35   0.05 -191.39 10.46   0.00        0.01     -0.01   \n",
      "2002-07-29   1.11 15.85   0.03 -298.09 35.35  -0.02        0.00     -0.02   \n",
      "2002-07-30   1.08 17.24  -0.00 -227.78 35.35  -0.04       -0.01     -0.03   \n",
      "2002-07-31   1.06 18.23  -0.02 -141.23 31.11  -0.04       -0.01     -0.03   \n",
      "2002-08-01   1.03 18.87  -0.04  -98.26 27.14  -0.05       -0.02     -0.03   \n",
      "2002-08-02   1.01 19.72  -0.06  -97.07 30.80  -0.05       -0.03     -0.03   \n",
      "2002-08-05   0.96 21.75  -0.07 -138.15 48.08  -0.07       -0.04     -0.03   \n",
      "2002-08-06   0.92 23.63  -0.10 -117.73 48.08  -0.08       -0.04     -0.03   \n",
      "2002-08-07   0.88 25.49  -0.12 -107.23 49.60  -0.09       -0.05     -0.03   \n",
      "2002-08-08   0.85 27.21  -0.14  -93.32 49.60  -0.09       -0.06     -0.03   \n",
      "2002-08-09   0.84 28.61  -0.16  -83.27 46.81  -0.09       -0.07     -0.02   \n",
      "2002-08-12   0.83 29.55  -0.17  -67.78 41.72  -0.09       -0.07     -0.02   \n",
      "2002-08-13   0.82 30.62  -0.17  -73.17 44.62  -0.09       -0.07     -0.02   \n",
      "2002-08-14   0.80 31.62  -0.17  -61.48 44.62  -0.09       -0.08     -0.01   \n",
      "2002-08-15   0.79 31.91  -0.16  -20.47 35.69  -0.08       -0.08     -0.01   \n",
      "2002-08-16   0.80 30.71  -0.15   57.64 15.04  -0.07       -0.08      0.00   \n",
      "2002-08-19   0.82 29.36  -0.14   81.68 11.81  -0.06       -0.08      0.01   \n",
      "2002-08-20   0.84 27.36  -0.12  142.19  1.35  -0.05       -0.07      0.02   \n",
      "2002-08-21   0.86 25.64  -0.10  129.97  3.25  -0.04       -0.06      0.02   \n",
      "2002-08-22   0.88 23.99  -0.08  118.11  2.60  -0.03       -0.06      0.03   \n",
      "...           ...   ...    ...     ...   ...    ...         ...       ...   \n",
      "2019-07-09 372.85 13.57  10.99   84.69 20.65   6.01        4.00      2.01   \n",
      "2019-07-10 375.21 14.72  10.81  114.79 29.57   6.27        4.45      1.82   \n",
      "2019-07-11 377.19 13.78  10.08   50.65  1.56   6.49        4.86      1.63   \n",
      "2019-07-12 378.13 12.93   9.78  100.43  1.87   6.47        5.18      1.28   \n",
      "2019-07-15 378.07 12.54  10.19   21.78  7.51   5.88        5.32      0.56   \n",
      "2019-07-16 377.53 13.15  10.19  -74.36 21.15   4.82        5.22     -0.40   \n",
      "2019-07-17 376.37 13.72   9.66  -85.97 21.15   3.88        4.95     -1.07   \n",
      "2019-07-18 370.32 14.53   8.85 -139.11 25.02   2.83        4.53     -1.70   \n",
      "2019-07-19 364.23 17.47   5.74 -345.28 55.69  -1.01        3.42     -4.43   \n",
      "2019-07-22 356.78 20.39   1.88 -241.33 58.38  -4.80        1.78     -6.58   \n",
      "2019-07-23 348.85 23.35  -2.78 -182.18 61.83  -8.08       -0.20     -7.89   \n",
      "2019-07-24 341.01 26.10  -7.64 -135.37 61.83 -10.82       -2.32     -8.50   \n",
      "2019-07-25 334.36 27.85 -11.23  -94.59 50.65 -12.00       -4.26     -7.74   \n",
      "2019-07-26 329.46 28.61 -14.51  -61.92 38.43 -12.10       -5.82     -6.28   \n",
      "2019-07-29 325.67 28.46 -17.21  -32.28 26.51 -11.30       -6.92     -4.38   \n",
      "2019-07-30 321.55 28.28 -19.86  -26.93 25.95 -10.79       -7.69     -3.10   \n",
      "2019-07-31 321.70 28.44 -22.14  -37.79 30.55 -10.81       -8.32     -2.49   \n",
      "2019-08-01 321.80 28.86 -23.92  -37.37 34.28 -10.93       -8.84     -2.09   \n",
      "2019-08-02 322.41 29.25 -26.23  -39.24 34.28 -11.19       -9.31     -1.88   \n",
      "2019-08-05 322.36 30.00 -28.20  -59.79 39.82 -11.31       -9.71     -1.60   \n",
      "2019-08-06 322.36 31.07 -27.26 -108.43 44.95 -12.17      -10.20     -1.97   \n",
      "2019-08-07 320.55 32.06 -25.48  -97.80 44.95 -12.51      -10.66     -1.85   \n",
      "2019-08-08 318.58 33.40 -23.30 -135.19 50.75 -13.10      -11.15     -1.95   \n",
      "2019-08-09 316.11 33.22 -20.29  -37.75 30.88 -12.48      -11.42     -1.06   \n",
      "2019-08-12 313.47 33.11 -18.25  -58.77 31.69 -12.41      -11.62     -0.80   \n",
      "2019-08-13 311.76 33.16 -16.87  -70.58 33.83 -12.07      -11.71     -0.36   \n",
      "2019-08-14 309.96 32.76 -16.37  -40.94 27.57 -11.54      -11.67      0.13   \n",
      "2019-08-15 307.96 33.04 -16.06 -108.02 36.72 -12.05      -11.75     -0.30   \n",
      "2019-08-16 306.61 33.84 -15.29 -150.25 44.20 -12.58      -11.91     -0.66   \n",
      "2019-08-19 306.13 34.21 -14.03  -92.93 39.04 -12.28      -11.99     -0.30   \n",
      "\n",
      "              MOM    ROC    ...      CDLHARAMI  CDLHARAMICROSS  CDLHIKKAKE  \\\n",
      "Date                        ...                                              \n",
      "2002-07-12   0.28  28.40    ...              0               0           0   \n",
      "2002-07-15   0.26  28.01    ...              0               0           0   \n",
      "2002-07-16   0.14  14.37    ...              0               0           0   \n",
      "2002-07-17   0.10   9.59    ...              0               0           0   \n",
      "2002-07-18  -0.08  -7.13    ...              0               0           0   \n",
      "2002-07-19  -0.07  -5.35    ...              0               0           0   \n",
      "2002-07-22  -0.12  -9.12    ...              0               0           0   \n",
      "2002-07-23  -0.00  -0.43    ...              0               0           0   \n",
      "2002-07-24  -0.01  -1.23    ...              0               0           0   \n",
      "2002-07-25  -0.05  -4.24    ...              0               0           0   \n",
      "2002-07-26  -0.23 -18.35    ...              0               0           0   \n",
      "2002-07-29  -0.34 -27.85    ...              0               0           0   \n",
      "2002-07-30  -0.27 -23.31    ...            100               0           0   \n",
      "2002-07-31  -0.21 -18.83    ...              0               0           0   \n",
      "2002-08-01  -0.14 -12.46    ...              0               0           0   \n",
      "2002-08-02  -0.26 -22.44    ...              0               0           0   \n",
      "2002-08-05  -0.38 -32.94    ...              0               0           0   \n",
      "2002-08-06  -0.40 -33.80    ...              0               0           0   \n",
      "2002-08-07  -0.39 -33.64    ...              0               0         100   \n",
      "2002-08-08  -0.32 -28.78    ...              0               0           0   \n",
      "2002-08-09  -0.24 -23.16    ...              0               0           0   \n",
      "2002-08-12  -0.08  -9.67    ...              0               0           0   \n",
      "2002-08-13  -0.12 -13.61    ...              0               0           0   \n",
      "2002-08-14  -0.14 -15.47    ...            100             100           0   \n",
      "2002-08-15  -0.12 -12.58    ...              0               0        -100   \n",
      "2002-08-16  -0.01  -0.96    ...              0               0           0   \n",
      "2002-08-19   0.13  17.36    ...              0               0           0   \n",
      "2002-08-20   0.21  26.73    ...              0               0           0   \n",
      "2002-08-21   0.19  24.41    ...              0               0           0   \n",
      "2002-08-22   0.18  22.05    ...            100               0           0   \n",
      "...           ...    ...    ...            ...             ...         ...   \n",
      "2019-07-09   6.95   1.88    ...              0               0           0   \n",
      "2019-07-10   8.89   2.40    ...              0               0           0   \n",
      "2019-07-11  20.70   5.75    ...              0               0           0   \n",
      "2019-07-12  17.30   4.78    ...              0               0           0   \n",
      "2019-07-15   3.23   0.87    ...              0               0           0   \n",
      "2019-07-16  -0.72  -0.20    ...              0               0           0   \n",
      "2019-07-17  -8.61  -2.30    ...              0               0           0   \n",
      "2019-07-18 -12.99  -3.46    ...              0               0         100   \n",
      "2019-07-19 -56.51 -14.80    ...              0               0           0   \n",
      "2019-07-22 -65.45 -17.20    ...              0               0           0   \n",
      "2019-07-23 -65.54 -17.42    ...              0               0           0   \n",
      "2019-07-24 -72.63 -19.12    ...              0               0           0   \n",
      "2019-07-25 -63.06 -16.55    ...              0               0        -100   \n",
      "2019-07-26 -53.04 -13.98    ...              0               0           0   \n",
      "2019-07-29 -37.47 -10.04    ...              0               0           0   \n",
      "2019-07-30 -33.90  -9.25    ...              0               0           0   \n",
      "2019-07-31 -40.06 -10.95    ...              0               0           0   \n",
      "2019-08-01 -39.45 -10.88    ...              0               0           0   \n",
      "2019-08-02  -5.71  -1.76    ...              0               0           0   \n",
      "2019-08-05   3.73   1.18    ...              0               0         100   \n",
      "2019-08-06  -2.99  -0.96    ...              0               0           0   \n",
      "2019-08-07   2.80   0.91    ...              0               0           0   \n",
      "2019-08-08 -13.65  -4.29    ...              0               0         100   \n",
      "2019-08-09 -10.56  -3.23    ...              0               0         200   \n",
      "2019-08-12 -26.85  -8.00    ...              0               0           0   \n",
      "2019-08-13 -21.87  -6.57    ...              0               0           0   \n",
      "2019-08-14 -13.65  -4.19    ...              0               0           0   \n",
      "2019-08-15 -23.88  -7.39    ...              0               0           0   \n",
      "2019-08-16 -23.74  -7.43    ...              0               0           0   \n",
      "2019-08-19 -16.03  -5.03    ...              0               0           0   \n",
      "\n",
      "            CDLINVERTEDHAMMER  CDLLONGLEGGEDDOJI  CDLMORNINGDOJISTAR  \\\n",
      "Date                                                                   \n",
      "2002-07-12                  0                  0                   0   \n",
      "2002-07-15                  0                  0                   0   \n",
      "2002-07-16                  0                  0                   0   \n",
      "2002-07-17                  0                  0                   0   \n",
      "2002-07-18                  0                  0                   0   \n",
      "2002-07-19                  0                  0                   0   \n",
      "2002-07-22                  0                  0                   0   \n",
      "2002-07-23                  0                  0                   0   \n",
      "2002-07-24                  0                  0                   0   \n",
      "2002-07-25                  0                  0                   0   \n",
      "2002-07-26                  0                  0                   0   \n",
      "2002-07-29                  0                  0                   0   \n",
      "2002-07-30                  0                  0                   0   \n",
      "2002-07-31                  0                  0                   0   \n",
      "2002-08-01                  0                  0                   0   \n",
      "2002-08-02                  0                  0                   0   \n",
      "2002-08-05                  0                  0                   0   \n",
      "2002-08-06                  0                100                   0   \n",
      "2002-08-07                  0                  0                   0   \n",
      "2002-08-08                  0                  0                   0   \n",
      "2002-08-09                  0                100                   0   \n",
      "2002-08-12                  0                  0                   0   \n",
      "2002-08-13                  0                  0                   0   \n",
      "2002-08-14                  0                100                   0   \n",
      "2002-08-15                  0                  0                   0   \n",
      "2002-08-16                  0                  0                   0   \n",
      "2002-08-19                  0                  0                   0   \n",
      "2002-08-20                  0                  0                   0   \n",
      "2002-08-21                  0                  0                   0   \n",
      "2002-08-22                  0                  0                   0   \n",
      "...                       ...                ...                 ...   \n",
      "2019-07-09                  0                  0                   0   \n",
      "2019-07-10                  0                  0                   0   \n",
      "2019-07-11                  0                  0                   0   \n",
      "2019-07-12                  0                  0                   0   \n",
      "2019-07-15                  0                  0                   0   \n",
      "2019-07-16                  0                  0                   0   \n",
      "2019-07-17                  0                  0                   0   \n",
      "2019-07-18                  0                  0                   0   \n",
      "2019-07-19                  0                  0                   0   \n",
      "2019-07-22                  0                  0                   0   \n",
      "2019-07-23                  0                  0                   0   \n",
      "2019-07-24                  0                  0                   0   \n",
      "2019-07-25                  0                  0                   0   \n",
      "2019-07-26                  0                  0                   0   \n",
      "2019-07-29                  0                  0                   0   \n",
      "2019-07-30                  0                  0                   0   \n",
      "2019-07-31                  0                  0                   0   \n",
      "2019-08-01                  0                  0                   0   \n",
      "2019-08-02                  0                  0                   0   \n",
      "2019-08-05                  0                  0                   0   \n",
      "2019-08-06                  0                  0                   0   \n",
      "2019-08-07                  0                100                   0   \n",
      "2019-08-08                  0                  0                   0   \n",
      "2019-08-09                  0                  0                   0   \n",
      "2019-08-12                  0                  0                   0   \n",
      "2019-08-13                  0                  0                   0   \n",
      "2019-08-14                  0                  0                   0   \n",
      "2019-08-15                  0                  0                   0   \n",
      "2019-08-16                  0                  0                   0   \n",
      "2019-08-19                  0                  0                   0   \n",
      "\n",
      "            CDLMORNINGSTAR  CDLSHOOTINGSTAR  CDLSPINNINGTOP  CDLTAKURI  \n",
      "Date                                                                    \n",
      "2002-07-12               0                0               0          0  \n",
      "2002-07-15               0                0               0          0  \n",
      "2002-07-16               0                0               0          0  \n",
      "2002-07-17               0                0               0          0  \n",
      "2002-07-18               0                0               0          0  \n",
      "2002-07-19               0                0               0          0  \n",
      "2002-07-22               0                0               0          0  \n",
      "2002-07-23               0                0               0          0  \n",
      "2002-07-24               0                0               0          0  \n",
      "2002-07-25               0                0               0          0  \n",
      "2002-07-26               0                0               0          0  \n",
      "2002-07-29               0                0               0          0  \n",
      "2002-07-30               0                0            -100          0  \n",
      "2002-07-31               0                0               0          0  \n",
      "2002-08-01               0                0               0          0  \n",
      "2002-08-02               0                0               0          0  \n",
      "2002-08-05               0                0               0          0  \n",
      "2002-08-06               0                0            -100          0  \n",
      "2002-08-07               0                0               0          0  \n",
      "2002-08-08               0                0               0          0  \n",
      "2002-08-09               0                0             100        100  \n",
      "2002-08-12               0                0               0          0  \n",
      "2002-08-13               0                0               0          0  \n",
      "2002-08-14               0                0             100          0  \n",
      "2002-08-15               0                0               0          0  \n",
      "2002-08-16               0                0               0          0  \n",
      "2002-08-19               0                0               0          0  \n",
      "2002-08-20               0                0               0          0  \n",
      "2002-08-21               0                0               0          0  \n",
      "2002-08-22               0                0               0          0  \n",
      "...                    ...              ...             ...        ...  \n",
      "2019-07-09               0                0               0          0  \n",
      "2019-07-10               0                0             100          0  \n",
      "2019-07-11               0                0               0          0  \n",
      "2019-07-12               0                0               0          0  \n",
      "2019-07-15               0                0               0          0  \n",
      "2019-07-16               0                0               0          0  \n",
      "2019-07-17               0                0               0          0  \n",
      "2019-07-18               0                0               0          0  \n",
      "2019-07-19               0                0             100          0  \n",
      "2019-07-22               0                0               0          0  \n",
      "2019-07-23               0                0            -100          0  \n",
      "2019-07-24               0                0               0          0  \n",
      "2019-07-25               0                0               0          0  \n",
      "2019-07-26               0                0               0          0  \n",
      "2019-07-29               0                0               0          0  \n",
      "2019-07-30               0                0               0          0  \n",
      "2019-07-31               0                0               0          0  \n",
      "2019-08-01               0                0            -100          0  \n",
      "2019-08-02               0                0               0          0  \n",
      "2019-08-05               0                0               0          0  \n",
      "2019-08-06               0                0               0          0  \n",
      "2019-08-07               0                0            -100          0  \n",
      "2019-08-08               0                0               0          0  \n",
      "2019-08-09               0                0               0          0  \n",
      "2019-08-12               0                0               0          0  \n",
      "2019-08-13               0                0               0          0  \n",
      "2019-08-14               0                0               0          0  \n",
      "2019-08-15               0                0               0          0  \n",
      "2019-08-16               0                0               0          0  \n",
      "2019-08-19               0                0               0          0  \n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4306 rows x 131 columns]\n"
     ]
    }
   ],
   "source": [
    "print(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now you COULD run feature selector again here to get rid of any\n",
    "# columns that have more than 0.60 of their values as the same,\n",
    "# but I'm not going to for now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] New Feature types:\n",
      "9MA                          float64\n",
      "ADX                          float64\n",
      "APO                          float64\n",
      "CCI                          float64\n",
      "DX                           float64\n",
      "MACD                         float64\n",
      "MACDSIGNAL                   float64\n",
      "MACDHIST                     float64\n",
      "MOM                          float64\n",
      "ROC                          float64\n",
      "RSI                          float64\n",
      "STOCHSLOWK                   float64\n",
      "STOCHSLOWD                   float64\n",
      "ULTOSC                       float64\n",
      "OBV                          float64\n",
      "ATR                          float64\n",
      "BETA                         float64\n",
      "LINEARREG_ANGLE              float64\n",
      "STDDEV                       float64\n",
      "LINEARREG <= PrevAdjClose      int64\n",
      "ULTOSC <= STOCHSLOWK           int64\n",
      "DX <= LowerBB                  int64\n",
      "20MA <= SAR                    int64\n",
      "ROC <= MOM                     int64\n",
      "DX <= ATR                      int64\n",
      "STDDEV <= MOM                  int64\n",
      "BETA <= MOM                    int64\n",
      "ATR <= APO                     int64\n",
      "ULTOSC <= RSI                  int64\n",
      "9MA <= Open                    int64\n",
      "                              ...   \n",
      "PrevLow <= TSF                 int64\n",
      "Open <= PrevLow                int64\n",
      "PrevHigh <= PrevOpen           int64\n",
      "UpperBB <= PrevAdjClose        int64\n",
      "Open <= PrevHigh               int64\n",
      "PrevOpen                     float64\n",
      "PrevHigh                     float64\n",
      "PrevLow                      float64\n",
      "PrevAdjClose                 float64\n",
      "PrevVol                      float64\n",
      "Open                         float64\n",
      "Adj Close                    float64\n",
      "CDLDOJI                        int64\n",
      "CDLDOJISTAR                    int64\n",
      "CDLDRAGONFLYDOJI               int64\n",
      "CDLEVENINGDOJISTAR             int64\n",
      "CDLEVENINGSTAR                 int64\n",
      "CDLGRAVESTONEDOJI              int64\n",
      "CDLHAMMER                      int64\n",
      "CDLHARAMI                      int64\n",
      "CDLHARAMICROSS                 int64\n",
      "CDLHIKKAKE                     int64\n",
      "CDLINVERTEDHAMMER              int64\n",
      "CDLLONGLEGGEDDOJI              int64\n",
      "CDLMORNINGDOJISTAR             int64\n",
      "CDLMORNINGSTAR                 int64\n",
      "CDLSHOOTINGSTAR                int64\n",
      "CDLSPINNINGTOP                 int64\n",
      "CDLTAKURI                      int64\n",
      "DailyResult                    int64\n",
      "Length: 131, dtype: object\n"
     ]
    }
   ],
   "source": [
    "# Now check the new types of the features, because you'll see now\n",
    "# that we have True and False boolean values, that we'll turn into numerical\n",
    "# values of 1 and 0 below, because machine learners like numbers not words.\n",
    "# Convert boolean True False datapoints to 1 and 0\n",
    "df = df.applymap(lambda x: 1 if x == True else x)\n",
    "df = df.applymap(lambda x: 0 if x == False else x)\n",
    "\n",
    "# Move dailyresult to the end\n",
    "dailyresult = df['DailyResult']\n",
    "df=df.drop(['DailyResult'],axis=1)\n",
    "df['DailyResult']=dailyresult\n",
    "\n",
    "# Now check to make sure our data is all numeric int's and float's.\n",
    "print('[INFO] New Feature types:')\n",
    "print(df.dtypes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's see the dataframe head again.\n",
    "#print(df.head())\n",
    "#print('Done!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9MA                          False\n",
      "ADX                          False\n",
      "APO                          False\n",
      "CCI                          False\n",
      "DX                           False\n",
      "MACD                         False\n",
      "MACDSIGNAL                   False\n",
      "MACDHIST                     False\n",
      "MOM                          False\n",
      "ROC                          False\n",
      "RSI                          False\n",
      "STOCHSLOWK                   False\n",
      "STOCHSLOWD                   False\n",
      "ULTOSC                       False\n",
      "OBV                          False\n",
      "ATR                          False\n",
      "BETA                         False\n",
      "LINEARREG_ANGLE              False\n",
      "STDDEV                       False\n",
      "LINEARREG <= PrevAdjClose    False\n",
      "ULTOSC <= STOCHSLOWK         False\n",
      "DX <= LowerBB                False\n",
      "20MA <= SAR                  False\n",
      "ROC <= MOM                   False\n",
      "DX <= ATR                    False\n",
      "STDDEV <= MOM                False\n",
      "BETA <= MOM                  False\n",
      "ATR <= APO                   False\n",
      "ULTOSC <= RSI                False\n",
      "9MA <= Open                  False\n",
      "                             ...  \n",
      "PrevLow <= TSF               False\n",
      "Open <= PrevLow              False\n",
      "PrevHigh <= PrevOpen         False\n",
      "UpperBB <= PrevAdjClose      False\n",
      "Open <= PrevHigh             False\n",
      "PrevOpen                     False\n",
      "PrevHigh                     False\n",
      "PrevLow                      False\n",
      "PrevAdjClose                 False\n",
      "PrevVol                      False\n",
      "Open                         False\n",
      "Adj Close                    False\n",
      "CDLDOJI                      False\n",
      "CDLDOJISTAR                  False\n",
      "CDLDRAGONFLYDOJI             False\n",
      "CDLEVENINGDOJISTAR           False\n",
      "CDLEVENINGSTAR               False\n",
      "CDLGRAVESTONEDOJI            False\n",
      "CDLHAMMER                    False\n",
      "CDLHARAMI                    False\n",
      "CDLHARAMICROSS               False\n",
      "CDLHIKKAKE                   False\n",
      "CDLINVERTEDHAMMER            False\n",
      "CDLLONGLEGGEDDOJI            False\n",
      "CDLMORNINGDOJISTAR           False\n",
      "CDLMORNINGSTAR               False\n",
      "CDLSHOOTINGSTAR              False\n",
      "CDLSPINNINGTOP               False\n",
      "CDLTAKURI                    False\n",
      "DailyResult                  False\n",
      "Length: 131, dtype: bool\n"
     ]
    }
   ],
   "source": [
    "# Now confirm if we have any nan values. All Falses? Great!\n",
    "print(pd.isnull(df).any())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This section has been added in to remove certain characters that XGBoost\n",
    "# doesn't recognize in the Feature/Column names or else we are going to get \n",
    "# an error after training occurs:\n",
    "# ValueError: feature_names may not contain [, ] or <\n",
    "import re\n",
    "regex = re.compile(r\"\\[|\\]|<\", re.IGNORECASE)\n",
    "df.columns = [regex.sub(\"_\", col) if any(x in str(col) for x in set(('[', ']'))) else col for col in df.columns.values]\n",
    "df.columns = [regex.sub(\"LESSTHAN\", col) if any(x in str(col) for x in set(('<'))) else col for col in df.columns.values]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. SPLIT THE DATASET FOR TRAINING/TESTING AND PREDICTIONS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of features:  (4306, 129)\n"
     ]
    }
   ],
   "source": [
    "# We're getting close to training our model! Congrats for\n",
    "# hanging on this far.\n",
    "\n",
    "# Split the dataset into features (X) and a target (Y)\n",
    "# and print the shapes of them\n",
    "X = df.drop(['Adj Close','DailyResult'], axis=1) # Everything but what we're trying to predict\n",
    "Y = df['DailyResult'] # This is what we're trying to predict\n",
    "\n",
    "print('Shape of features: ', X.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of target:  (4306,)\n"
     ]
    }
   ],
   "source": [
    "print('Shape of target: ', Y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Standardize the data. Commenting this out until I can figure out how to\n",
    "# unscale the prediction dataset for review later. This helps with model\n",
    "# training performance and other reasons, but be careful because if you standardize a training\n",
    "# dataset, then go to use the model on a new prediction dataset, if that \n",
    "# new dataset has values greater or less than your standarized scale,\n",
    "# the model might not be as accurate. Maybe scale to a MinMax of having\n",
    "# a 10-20% buffer above/below???\n",
    "#from sklearn.preprocessing import StandardScaler, MinMaxScaler\n",
    "#scaler = MinMaxScaler().fit(X)\n",
    "#scaled_X = scaler.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split dataset into training and testing/validation data. Think of this as,\n",
    "# the model learns from the testing data, then tests to see how well it did\n",
    "# (internally) on the testing data. Much like what we're going to do with the\n",
    "# model on our prediction dataset later.\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "seed = 9 # For reproducibility of the data in future steps\n",
    "test_size = 0.20 # 80% training, 20% testing\n",
    "\n",
    "# This also randomly shuffles the data. We're going to disable this because we \n",
    "# are dealing with a time series problem. We don't want our model to just\n",
    "# recognize an overall uptrend in our data though, so we're going to employ\n",
    "# cross validation in a minute to combat this.\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=test_size, random_state=seed) # , shuffle=False\n",
    "\n",
    "# Now, becuase we're using many different types of regression algorithms in\n",
    "# TPOT, we need to make sure that every one of them can understand our data\n",
    "# in the same way, so long story short, convert the training features, and \n",
    "# testing features to Numpy arrays, but leave the Y's as a DF. It's a long\n",
    "# story and I had many errors to deal with in figuring this portion out.\n",
    "# See https://github.com/EpistasisLab/tpot/issues/738\n",
    "# or https://stackoverflow.com/questions/57488274/feature-names-mismatch-when-passing-x-test-to-predict-function-again-still\n",
    "# ...if you want.\n",
    "X_train=X_train.values\n",
    "X_test=X_test.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Printing the shapes of the training/testing feature/label sets...\n",
      "(3444, 129)\n",
      "(862, 129)\n",
      "(3444,)\n",
      "(862,)\n"
     ]
    }
   ],
   "source": [
    "print('[INFO] Printing the shapes of the training/testing feature/label sets...')\n",
    "print(X_train.shape)\n",
    "print(X_test.shape)\n",
    "print(Y_train.shape)\n",
    "print(Y_test.shape)\n",
    "\n",
    "\n",
    "# I know it's tedious to keep printing the data and shapes like this, but the\n",
    "# further along you get in your coding, the more important it becomes to know\n",
    "# exactly what type of data is being passed from step to step to help mitigate\n",
    "# and pinpoint where you might run into any errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 8. TRAIN A MODEL USING TPOT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The time is finally here! Time to train a regression model. Think...\n",
    "\n",
    "regression = predicting a number\n",
    "\n",
    "classification = predicting a category\n",
    "\n",
    "Since we're predicting the daily closing price, we need regression.\n",
    "TPOT is a module that automates the process of finding the best\n",
    "machine learning model (or stack of models) for your dataset, with \n",
    "the best hyperparameter(s) for each one, a process that would \n",
    "normally need to be done manually. This automated machine learning\n",
    "is a no-brainer. Let it do the magic for you so you don't have to.\n",
    "Notice how I like the idea of automation, like feature engineering,\n",
    "selection, regression analysis, etc.? It just makes sense, even if\n",
    "it's a little more computationally taxing (although we try to do it \n",
    "as efficiently as possible).\n",
    "\n",
    "The way I describe this part is, you have some specific music (data), and\n",
    "you're trying to pick the best sound system (regression pipeline)\n",
    "to get the best sound (prediction). Each of the amplifiers (algorithms)\n",
    "have a set of dials (hyperparameters) that need adjusting and tweaking\n",
    "to get right. And maybe stacking multiple amplifiers would make the music sound\n",
    "even better! So what TPOT does is it trains a model to get the best possible\n",
    "result (i.e. lowest loss function) by trying many, MANY different combinations\n",
    "of amplifiers and dial settings to end up with the best possible sound system.\n",
    "Make sense? Let's do it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\deap\\tools\\_hypervolume\\pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow.\n",
      "  \"module. Expect this to be very slow.\", ImportWarning)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n",
      "C:\\Users\\windowshopr\\AppData\\Local\\Programs\\Python\\Python36\\lib\\importlib\\_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "11cb6611269d49ef913401b040727d45",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, description='Optimization Progress', max=420, style=ProgressStyle(descript…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generation 1 - Current best internal CV score: 0.518874281331773\n",
      "Generation 2 - Current best internal CV score: 0.518874281331773\n",
      "Generation 3 - Current best internal CV score: 0.5214753690006988\n",
      "Generation 4 - Current best internal CV score: 0.5214753690006988\n",
      "Generation 5 - Current best internal CV score: 0.5214753690006988\n",
      "Generation 6 - Current best internal CV score: 0.5214753690006988\n",
      "Generation 7 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 8 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 9 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 10 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 11 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 12 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 13 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 14 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 15 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 16 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 17 - Current best internal CV score: 0.5301761160913883\n",
      "Generation 18 - Current best internal CV score: 0.5301761160913883\n",
      "\n",
      "\n",
      "TPOT closed during evaluation in one generation.\n",
      "WARNING: TPOT may not provide a good pipeline if TPOT is stopped/interrupted in a early generation.\n",
      "\n",
      "\n",
      "TPOT closed prematurely. Will use the current best pipeline.\n",
      "\n",
      "Best pipeline: GradientBoostingClassifier(Binarizer(input_matrix, threshold=0.8), learning_rate=0.5, max_depth=4, max_features=0.25, min_samples_leaf=13, min_samples_split=18, n_estimators=100, subsample=0.9500000000000001)\n"
     ]
    }
   ],
   "source": [
    "from tpot import TPOTClassifier\n",
    "\n",
    "# I made a custom config dictionary for TPOT to use.\n",
    "# I've made this list full of Regressors that use the\n",
    "# .feature_importances_ attribute of sklearn to review\n",
    "# what the feature imporances are once the model is run.\n",
    "\n",
    "# This is the list of regressors and pre-processors\n",
    "# that I've used. You can add or subtract to this list,\n",
    "# but you may end up running into errors when it comes to displaying\n",
    "# the feature importances part that you'll have to deal with.\n",
    "# Read here for more: https://epistasislab.github.io/tpot/using/#customizing-tpots-operators-and-parameters\n",
    "\n",
    "# Define the protecting function.\n",
    "def train_function(X_train, Y_train):\n",
    "    warnings.simplefilter(action='once', category=FutureWarning)\n",
    "    warnings.simplefilter(action='once', category=UserWarning)\n",
    "    warnings.simplefilter(action='once', category=DeprecationWarning)\n",
    "    warnings.simplefilter(action='once', category=ImportWarning)\n",
    "    \n",
    "    tpot_config = {\n",
    "\n",
    "    'xgboost.XGBClassifier': {\n",
    "        'n_estimators': [100],\n",
    "        'max_depth': range(1, 11),\n",
    "        'learning_rate': [1e-3, 1e-2, 1e-1, 0.5, 1.],\n",
    "        'subsample': np.arange(0.05, 1.01, 0.05),\n",
    "        'min_child_weight': range(1, 21),\n",
    "        'nthread': [1]\n",
    "    },\n",
    "        \n",
    "\n",
    "    'sklearn.tree.DecisionTreeClassifier': {\n",
    "        'criterion': [\"gini\", \"entropy\"],\n",
    "        'max_depth': range(1, 11),\n",
    "        'min_samples_split': range(2, 21),\n",
    "        'min_samples_leaf': range(1, 21)\n",
    "    },\n",
    "\n",
    "    'sklearn.ensemble.ExtraTreesClassifier': {\n",
    "        'n_estimators': [100],\n",
    "        'criterion': [\"gini\", \"entropy\"],\n",
    "        'max_features': np.arange(0.05, 1.01, 0.05),\n",
    "        'min_samples_split': range(2, 21),\n",
    "        'min_samples_leaf': range(1, 21),\n",
    "        'bootstrap': [True, False]\n",
    "    },\n",
    "\n",
    "    'sklearn.ensemble.RandomForestClassifier': {\n",
    "        'n_estimators': [100],\n",
    "        'criterion': [\"gini\", \"entropy\"],\n",
    "        'max_features': np.arange(0.05, 1.01, 0.05),\n",
    "        'min_samples_split': range(2, 21),\n",
    "        'min_samples_leaf':  range(1, 21),\n",
    "        'bootstrap': [True, False]\n",
    "    },\n",
    "\n",
    "    'sklearn.ensemble.GradientBoostingClassifier': {\n",
    "        'n_estimators': [100],\n",
    "        'learning_rate': [1e-3, 1e-2, 1e-1, 0.5, 1.],\n",
    "        'max_depth': range(1, 11),\n",
    "        'min_samples_split': range(2, 21),\n",
    "        'min_samples_leaf': range(1, 21),\n",
    "        'subsample': np.arange(0.05, 1.01, 0.05),\n",
    "        'max_features': np.arange(0.05, 1.01, 0.05)\n",
    "    },\n",
    "\n",
    "\n",
    "    # Preprocesssors\n",
    "    'sklearn.preprocessing.Binarizer': {\n",
    "        'threshold': np.arange(0.0, 1.01, 0.05)\n",
    "    },\n",
    "\n",
    "    'sklearn.decomposition.FastICA': {\n",
    "        'tol': np.arange(0.0, 1.01, 0.05)\n",
    "    },\n",
    "\n",
    "    'sklearn.cluster.FeatureAgglomeration': {\n",
    "        'linkage': ['ward', 'complete', 'average'],\n",
    "        'affinity': ['euclidean', 'l1', 'l2', 'manhattan', 'cosine']\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.MaxAbsScaler': {\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.MinMaxScaler': {\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.Normalizer': {\n",
    "        'norm': ['l1', 'l2', 'max']\n",
    "    },\n",
    "\n",
    "    'sklearn.kernel_approximation.Nystroem': {\n",
    "        'kernel': ['rbf', 'cosine', 'chi2', 'laplacian', 'polynomial', 'poly', 'linear', 'additive_chi2', 'sigmoid'],\n",
    "        'gamma': np.arange(0.0, 1.01, 0.05),\n",
    "        'n_components': range(1, 11)\n",
    "    },\n",
    "\n",
    "    'sklearn.decomposition.PCA': {\n",
    "        'svd_solver': ['randomized'],\n",
    "        'iterated_power': range(1, 11)\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.PolynomialFeatures': {\n",
    "        'degree': [2],\n",
    "        'include_bias': [False],\n",
    "        'interaction_only': [False]\n",
    "    },\n",
    "\n",
    "    'sklearn.kernel_approximation.RBFSampler': {\n",
    "        'gamma': np.arange(0.0, 1.01, 0.05)\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.RobustScaler': {\n",
    "    },\n",
    "\n",
    "    'sklearn.preprocessing.StandardScaler': {\n",
    "    },\n",
    "\n",
    "    'tpot.builtins.ZeroCount': {\n",
    "    },\n",
    "\n",
    "    'tpot.builtins.OneHotEncoder': {\n",
    "        'minimum_fraction': [0.05, 0.1, 0.15, 0.2, 0.25],\n",
    "        'sparse': [False],\n",
    "        'threshold': [10]\n",
    "    },\n",
    "\n",
    "\n",
    " # Selectors\n",
    "    'sklearn.feature_selection.SelectFwe': {\n",
    "        'alpha': np.arange(0, 0.05, 0.001),\n",
    "        'score_func': {\n",
    "            'sklearn.feature_selection.f_classif': None\n",
    "        }\n",
    "    },\n",
    "\n",
    "    'sklearn.feature_selection.SelectPercentile': {\n",
    "        'percentile': range(1, 100),\n",
    "        'score_func': {\n",
    "            'sklearn.feature_selection.f_classif': None\n",
    "        }\n",
    "    },\n",
    "\n",
    "    'sklearn.feature_selection.VarianceThreshold': {\n",
    "        'threshold': [0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05, 0.1, 0.2]\n",
    "    },\n",
    "\n",
    "    'sklearn.feature_selection.RFE': {\n",
    "        'step': np.arange(0.05, 1.01, 0.05),\n",
    "        'estimator': {\n",
    "            'sklearn.ensemble.ExtraTreesClassifier': {\n",
    "                'n_estimators': [100],\n",
    "                'criterion': ['gini', 'entropy'],\n",
    "                'max_features': np.arange(0.05, 1.01, 0.05)\n",
    "            }\n",
    "        }\n",
    "    },\n",
    "\n",
    "    'sklearn.feature_selection.SelectFromModel': {\n",
    "        'threshold': np.arange(0, 1.01, 0.05),\n",
    "        'estimator': {\n",
    "            'sklearn.ensemble.ExtraTreesClassifier': {\n",
    "                'n_estimators': [100],\n",
    "                'criterion': ['gini', 'entropy'],\n",
    "                'max_features': np.arange(0.05, 1.01, 0.05)\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "\n",
    "\n",
    "}\n",
    "\n",
    "# Now comes another important aspect of the training/testing phase;\n",
    "# Cross Validation. CV is action of replacing the testing portion\n",
    "# of your training/testing dataset with a new section within that\n",
    "# same dataset. So in our example, we are using a 75/25 split.\n",
    "# During the first \"fold\" of our CV run, we train on that 75 \n",
    "# data, and test on the 25. Then in the next fold, we take a random\n",
    "# 25% data from the 75% data and swap it out with the current 25%.\n",
    "# This is important because it's a way to prevent \"over-fitting\" our\n",
    "# model to our data. If we were training a model on Amazon's stock prices,\n",
    "# without randomly shuffling the testing/training data, and without CV,\n",
    "# eventually our model would learn to just buy and hold that stock to \n",
    "# get the best result, but that doesn't mean the model has good\n",
    "# predictive capabilities because it wouldn't know what to do when it \n",
    "# starts going down. CV helps to truncate the full historical data to \n",
    "# train on various sections within the data. More here:\n",
    "\n",
    "# https://towardsdatascience.com/overfitting-vs-underfitting-a-complete-example-d05dd7e19765\n",
    "\n",
    "# TPOT does this cross validation for us already. Yay!\n",
    "\n",
    "\n",
    "# Cross Validation folds to run\n",
    "    folds   = 10 # Default is 10\n",
    "\n",
    "    #earlystop = 30 # could have this feature added in if you want, I don't for now)\n",
    "\n",
    "\n",
    "# Define the TPOT Classifier.\n",
    "# It's nice to have Dask installed/running during this part. Dask utilizes\n",
    "# parallel computing, meaning dividing the same task across a number of \n",
    "# \"workers\" (i.e. computer processors) to get a job done faster!\n",
    "# If you don't want to use it, make it false, and get rid of the n_jobs parameter.\n",
    "# n_jobs = -1 means use all available workers. Can set to 1, 2, 300, depending on\n",
    "# how many processors you have. See the TPOT API for more info :D\n",
    "\n",
    "# Also, review what the generations and population sizes mean when it comes to how\n",
    "# many regressors are going to be used during the optimization process. Again, look here:\n",
    "# https://epistasislab.github.io/tpot/api/#classification\n",
    "    best_model = TPOTClassifier(generations=20, population_size=20,\n",
    "                          offspring_size=None,\n",
    "                          scoring='accuracy', cv=folds,\n",
    "                          #n_jobs=1,\n",
    "                          max_time_mins=None,\n",
    "                          random_state=seed, config_dict=tpot_config,\n",
    "                          use_dask=True, # See n_jobs number\n",
    "                          verbosity=2)\n",
    "\n",
    "# Fit the TPOT regression to our training data to find the best model. Woohoo!\n",
    "    best_model.fit(X_train, Y_train)\n",
    "\n",
    "# Export the TPOT pipeline if you want to use it for anything later\n",
    "    if os.path.exists('./exported pipelines'):\n",
    "        pass\n",
    "    else:\n",
    "        os.mkdir('./exported pipelines')\n",
    "    best_model.export('./exported pipelines/' + ticker_input + '-prediction-pipeline.py')\n",
    "    \n",
    "    best_model = best_model\n",
    "    return(best_model)\n",
    "\n",
    "# Now run the function!\n",
    "best_model = train_function(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] TPOTs best pipeline is:\n",
      "GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
      "                           learning_rate=0.5, loss='deviance', max_depth=4,\n",
      "                           max_features=0.25, max_leaf_nodes=None,\n",
      "                           min_impurity_decrease=0.0, min_impurity_split=None,\n",
      "                           min_samples_leaf=13, min_samples_split=18,\n",
      "                           min_weight_fraction_leaf=0.0, n_estimators=100,\n",
      "                           n_iter_no_change=None, presort='auto',\n",
      "                           random_state=None, subsample=0.9500000000000001,\n",
      "                           tol=0.0001, validation_fraction=0.1, verbose=0,\n",
      "                           warm_start=False)\n",
      "We are done training the model! You can view the feature importances plot in your directory now.\n",
      "It is now time to predict the values in the testing/validation set and see how well it did internally...\n"
     ]
    }
   ],
   "source": [
    "print('[INFO] TPOTs best pipeline is:')\n",
    "print(best_model.fitted_pipeline_.steps[-1][1])\n",
    "print('We are done training the model! You can view the feature importances plot in your directory now.')\n",
    "print('It is now time to predict the values in the testing/validation set and see how well it did internally...')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[INFO] Printing the shape of X_test for debugging the predict function...\n",
      "(862, 129)\n"
     ]
    }
   ],
   "source": [
    "print('[INFO] Printing the shape of X_test for debugging the predict function...')\n",
    "#print(X_test[:,:])\n",
    "print(X_test.shape)\n",
    "\n",
    "# Now let's make sure X_test columns are in the same layout as X_train...\n",
    "#X_test = X_test[X_train.columns]\n",
    "#print(X_test.shape) # for debugging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 9. TEST THE MODEL ON THE TESTING/VALIDATION DATASET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=============================\n",
      "TPOT's final score on testing dataset is :  0.5232018561484919\n",
      "=============================\n",
      "[INFO] Accuracy Score on test set : 0.523\n",
      "[INFO] Balanced Accuracy Score on test set : 0.522\n"
     ]
    }
   ],
   "source": [
    "# This is to see how well it did internally using its testing data (25%).\n",
    "# Make predictions using the tuned model and display error metrics,\n",
    "from sklearn.metrics import accuracy_score, balanced_accuracy_score\n",
    "# Now make the predictions using our model and display the metrics\n",
    "predictions = best_model.predict(X_test)\n",
    "\n",
    "\n",
    "print('=============================')\n",
    "print(\"TPOT's final score on testing dataset is : \", best_model.score(X_test, Y_test))\n",
    "print('=============================')\n",
    "print(\"[INFO] Accuracy Score on test set : {}\".format(round(accuracy_score(Y_test, predictions), 3)))\n",
    "print(\"[INFO] Balanced Accuracy Score on test set : {}\".format(round(balanced_accuracy_score(Y_test, predictions), 3)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Date\n",
      "2004-07-07    0\n",
      "2005-04-21    1\n",
      "2018-10-08    1\n",
      "2003-10-14    1\n",
      "2007-07-18    0\n",
      "2016-09-23    1\n",
      "2013-01-07    1\n",
      "2010-09-01    1\n",
      "2007-01-12    0\n",
      "2017-11-02    1\n",
      "2010-09-08    1\n",
      "2008-10-13    1\n",
      "2003-02-20    0\n",
      "2008-04-21    1\n",
      "2003-09-04    1\n",
      "2009-05-12    1\n",
      "2018-10-03    0\n",
      "2013-08-15    0\n",
      "2017-06-09    0\n",
      "2006-09-13    1\n",
      "2019-01-23    0\n",
      "2010-06-30    0\n",
      "2016-02-23    0\n",
      "2019-05-22    1\n",
      "2008-08-11    1\n",
      "2008-05-19    0\n",
      "2009-11-11    1\n",
      "2017-05-15    0\n",
      "2011-05-17    1\n",
      "2008-08-01    0\n",
      "             ..\n",
      "2018-12-31    1\n",
      "2012-04-18    1\n",
      "2015-06-24    0\n",
      "2012-05-23    1\n",
      "2003-05-13    1\n",
      "2005-01-25    0\n",
      "2011-07-25    1\n",
      "2011-04-21    1\n",
      "2016-02-22    1\n",
      "2017-02-06    1\n",
      "2017-04-26    0\n",
      "2005-02-02    0\n",
      "2011-05-13    1\n",
      "2015-06-04    1\n",
      "2003-12-04    0\n",
      "2008-06-17    0\n",
      "2010-02-12    1\n",
      "2010-06-25    1\n",
      "2019-05-02    1\n",
      "2015-11-30    0\n",
      "2004-06-02    0\n",
      "2004-06-17    0\n",
      "2005-03-04    1\n",
      "2008-09-04    0\n",
      "2010-06-29    0\n",
      "2005-12-16    1\n",
      "2006-10-03    1\n",
      "2017-10-06    1\n",
      "2015-09-16    1\n",
      "2012-05-08    0\n",
      "Name: DailyResult, Length: 862, dtype: int64, array([0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0,\n",
      "       0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0,\n",
      "       1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0,\n",
      "       0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1,\n",
      "       1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0,\n",
      "       0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0,\n",
      "       0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1,\n",
      "       0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0,\n",
      "       0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0,\n",
      "       1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1,\n",
      "       0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0,\n",
      "       1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0,\n",
      "       0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1,\n",
      "       1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1,\n",
      "       0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1,\n",
      "       1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0,\n",
      "       0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1,\n",
      "       0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1,\n",
      "       1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0,\n",
      "       0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1,\n",
      "       0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1,\n",
      "       0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0,\n",
      "       0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1,\n",
      "       1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1,\n",
      "       1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1,\n",
      "       0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0,\n",
      "       0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1,\n",
      "       0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0,\n",
      "       1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0,\n",
      "       0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0,\n",
      "       1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0,\n",
      "       0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1,\n",
      "       1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0,\n",
      "       1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1,\n",
      "       0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1,\n",
      "       0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0,\n",
      "       1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1,\n",
      "       0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0,\n",
      "       0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0,\n",
      "       0, 0, 1, 0], dtype=int64)]\n"
     ]
    }
   ],
   "source": [
    "print([Y_test,predictions])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--\n",
      "Done predicting the testing set.\n"
     ]
    }
   ],
   "source": [
    "# Now create another lovely plot for us to save.\n",
    "# Plot between predictions and Y_test\n",
    "#x_axis = np.array(range(0, predictions.shape[0]))\n",
    "#plt.figure(figsize=(40,20)) # again adjust this size as needed\n",
    "#plt.plot(x_axis, predictions, linestyle=\"--\", marker=\"o\", alpha=0.7, color='r', label=\"predictions\")\n",
    "#plt.plot(x_axis, Y_test, linestyle=\"--\", marker=\"o\", alpha=0.7, color='g', label=\"Y_test\")\n",
    "#plt.xlabel('Row number')\n",
    "#plt.ylabel('PRICE')\n",
    "#plt.title('Predictions vs Y_test')\n",
    "#plt.legend(loc='lower right')\n",
    "#plt.savefig(\"./plots/predictions_vs_ytest.png\")\n",
    "#plt.clf()\n",
    "#plt.close()\n",
    "print('--')\n",
    "print('Done predicting the testing set.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "#print(prediction_df.columns) # for debugging purposes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Now fit the best pipeline to the training data to find feature importances...\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "# Extract what the best pipeline was and fit it to the training set\n",
    "# to get an idea of the most important features used by the model were.\n",
    "# Is there a way to do this during the first .fit() function so that\n",
    "# we don't have to run it a second time here? Althought it's not a big deal.\n",
    "print('Now fit the best pipeline to the training data to find feature importances...')\n",
    "exctracted_best_model = best_model.fitted_pipeline_.steps[-1][1]\n",
    "\n",
    "# Train only the `exctracted_best_model` using the training/vildation set\n",
    "exctracted_best_model.fit(X_train, Y_train)\n",
    "\n",
    "# plot model's feature importance and save the plot for later\n",
    "feature_importance = exctracted_best_model.feature_importances_\n",
    "feature_importance = 100.0 * (feature_importance / feature_importance.max())\n",
    "sorted_idx = np.argsort(feature_importance)\n",
    "pos        = np.arange(sorted_idx.shape[0]) + .5\n",
    "plt.figure(figsize=(40,20)) # play with this to adjust the size of the plot to your specs\n",
    "plt.barh(pos, feature_importance[sorted_idx], align='center')\n",
    "plt.yticks(pos, df.columns[sorted_idx])\n",
    "plt.xlabel('Relative Importance')\n",
    "plt.title('Variable Importance')\n",
    "plt.savefig(\"./plots/feature_importance.png\")\n",
    "plt.clf()\n",
    "plt.close()\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we get an accuracy of 52% from our model. Is that any good? A homework assignment might be to go back through the testing set, and if we received a Buy signal, calculate the price change between the Adj Close and the Open for that day. Did you in fact close higher than the open? Do this for all the days in the test set and see if you would have made any money. Remember, the testing and training sets are shuffled, so if you wanted to get a timeline return of this model, you'd have to go back and withhold a \"prediction\" dataset before passing it to the train test split function. You can see how this is done from my first project here:\n",
    "\n",
    "https://github.com/windowshopr/Project-1-Closing-Price-Stock-Regressor-Version-1.0\n",
    "\n",
    "# Thanks for reading! Future projects will be coming later."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}