n3a_writing_tests.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "source": [
    "# Testing Software\n",
    "\n",
    "⏱️ 30 min.\n",
    "\n",
    "Writing code is step one. Step two is making sure that the code you're writing is correct. \n",
    "\n",
    "Of course, this is a process that you have gone through when using Python (or Excel) to complete a process. When programming, it's best practice to encode these checks into `Tests`. \n",
    "\n",
    "Software testing is a tool we can use to:\n",
    "1. Build confidence that a process is automated correctly\n",
    "2. Allow us to make changes to the process without breaking it\n",
    "3. Help other people understand expected behavior of our automation\n",
    "\n",
    "If a process is something you're using on a go-forward basis, you probably want some tests for it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unit Tests\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unit tests are just tests for specific chunks (aka units!) of code. A unit test ensures that the specific ode performs as expected and handles various scenarios correctly.\n",
    "\n",
    "Consider the `add(x, y)` function below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def add(x, y):\n",
    "    return x + y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "💡 A **function** is often the most useful boundry to unit test on. If you've broken your functions up effectively, then each function should be doing a single, coherent chunk of work. \n",
    "\n",
    "As such, it makes sense to test the behavior of the function as a single unit!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Test a single addition works\n",
    "add(1, 2) == 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`assert` is a Python keyword that checks that the value passed to it is True, and optionally allows you to give an error message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "assert True, 'Asserting True should pass. Nothing should be printed after this cell.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "assert False, 'Asserting False should fail. We should get an error message after we run this cell.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "💡 When we test functions, we want to test:\n",
    "- The most common behaviors\n",
    "- Edge cases that we are likely to run into\n",
    "\n",
    "Overall, our goal isn't to write tests that pass easily. **Our goal is to write tests that probe the function, and ensure that is correct**. Don't just phone it in on tests. If you write good tests, you should be nervous to run them because they actually might fail and catch your bugs!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Test common cases\n",
    "assert add(1, 2) == 3, 'Adding positive numbers failed'\n",
    "assert add(-1, -2) == -3, 'Adding negative numbers failed'\n",
    "assert add(-1, 2) == 1, 'Adding a negative and positive number failed'\n",
    "\n",
    "# Some edge cases\n",
    "assert add(1, 2.2) == 3.2, 'Adding an integer and float failed'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🧑‍💻 Write a test that tests another case you're likely to encounter. Anything we might be missing with our tests?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# TODO: Write your new test case here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Input Data Validation\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the famous software saying goes: Garbage In, Garbage Out. \n",
    "\n",
    "Data validation tests ensure that we're not getting garbage in! \n",
    "\n",
    "Data validation tests looks very similar to unit tests, except they are testing your data rather than your functions!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "loans = pd.read_csv('data3_loans.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's add a simple check that we have the correct columns in our dataset, in the correct order. \n",
    "\n",
    "**One of the most common bugs in any data processing script is missing columns. By checking that the correct columns exist, and are in the correct order up-front, we can avoid much harder to understand and debug issues later on.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "assert loans.columns.tolist() == ['issue_date','income_category','annual_income','loan_amount','term','purpose','interest_payments','loan_condition','interest_rate','total_pymnt','total_rec_prncp'], 'Incorrect columns'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we are checking that simple values are equal to eachother (e.g. `1 + 2 == 3`), we can just use the `==` operator. However, with `pandas` objects, we need to do a bit more work to make sure that values are equal. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# This should fail\n",
    "df1 = pd.DataFrame({'A': [1, 2, 3]})\n",
    "df2 = pd.DataFrame({'A': [1, 2, 3]})\n",
    "assert df1 == df2, 'This should fail. Dataframes cannot be compared with =='"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# This should pass\n",
    "df1 = pd.DataFrame({'A': [1, 2, 3]})\n",
    "df2 = pd.DataFrame({'A': [1, 2, 3]})\n",
    "assert df1.equals(df2), \"The dataframes are not equal\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also make assertions about the actual _contents_ of our data. Let's imagine that the loans data we're processing is from before the year 2011. We can create a check for this easily."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "assert (pd.to_datetime(loans['issue_date'], format='mixed') < pd.to_datetime('1-1-2012')).all(), 'There is a date from after 2011.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, there is a date from after 2011! Again, catching this now, before processing our data, will result in less weird error messages, or in some cases stop a totally incorrect process from running! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🧑‍💻 Let's write a few tests that validate the structure of our data:\n",
    "- Check that the `income_category` is Low, Medium or High\n",
    "- Check that there are no nan values in `loan_amount`\n",
    "- Think of another reasonable test to make of this data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# TODO: Write your new tests here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Material vs. immaterial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sadly, floating point numbers in Python don't always play nice with our asserts. Check out the following assert that seems like it should be true!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "assert .1 + .1 + .1 == .3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This occurs because floating point numbers are not exactly represented internally within computers. This is not an issue for most use cases, but it does mean we have to take special care when writing tests to assert that there are no _material_ differences in our expected and actual values.\n",
    "\n",
    "Use the `math.isclose` function to check floating point numbers are close. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import math\n",
    "assert math.isclose(.1 + .1 + .1, .3, abs_tol=1e-6), 'This should not print. Numbers are equal.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to compare numbers within some tolerance across entire dataframes, we can use the `assert_frame_equal` utility function given to us by the `pandas` library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create the first DataFrame\n",
    "df1 = pd.DataFrame({'A': [1.0, 2.0, 3.0], 'B': [0.1, 0.2, 0.3]})\n",
    "\n",
    "# Create the second DataFrame with a slight difference in floating-point values\n",
    "df2 = pd.DataFrame({'A': [1.0, 2.0, 3.0], 'B': [0.100001, 0.2, 0.300001]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Compare the dataframes directly. This should error.\n",
    "assert df1.equals(df2), \"The dataframes are not equal\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Compare the DataFrames while ignoring small differences in floating-point numbers. By default, \n",
    "# checks that numbers are within 1e-8 of eachother, but this can be configured by the atol={tolerance} \n",
    "# parameter\n",
    "\n",
    "from pandas.testing import assert_frame_equal\n",
    "assert_frame_equal(df1, df2, atol=1e-8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Integration Testing\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The most expressive test you can build is an integration test. An integration test ensures that the fully-integrated process, from start to finish, works as you expect it to. \n",
    "\n",
    "Let's imagine we've used Python to automate the calculation of compounding interest with a `calculate_compound_interest` function. This is our full process, so we can write some integration tests to check it performs correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def calculate_compound_interest(principal, interest_rate, time):\n",
    "    \"\"\"\n",
    "    Calculate compound interest given the principal, interest rate, and time.\n",
    "\n",
    "    Args:\n",
    "        principal (float): The initial amount of money.\n",
    "        interest_rate (float): The interest rate per period.\n",
    "        time (int): The number of periods.\n",
    "\n",
    "    Returns:\n",
    "        float: The total amount after compounding the interest.\n",
    "    \"\"\"\n",
    "    amount = principal * (1 + interest_rate) ** time\n",
    "    return amount"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "# Test with principal = $1000, interest_rate = 0.05, time = 2\n",
    "assert math.isclose(calculate_compound_interest(1000, 0.05, 2), 1102.5)\n",
    "\n",
    "# Test with principal = $5000, interest_rate = 0.08, time = 5\n",
    "assert math.isclose(calculate_compound_interest(5000, 0.08, 5), 7346.640384000003)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🧑‍💻 Let's write a few more integration tests. As with unit tests, try to test both common cases, and edge cases. Some ideas:\n",
    "1. What if the number of periods is 0?\n",
    "2. What if the interest rate is 0, or negative?\n",
    "\n",
    "In all of these cases, creating _expected_ results might be the challenging part -- maybe use Excel?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# TODO: add your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Expected Results\n",
    "\n",
    "As you've likely noticed from the above integration tests, you need to have some _expected_ values to test against to write effective tests. For processes that are way more complex than calculating compound interest, you likely cannot turn to another tool to generate expected values easily. So what do?\n",
    "\n",
    "Since we're automating processes using Python, the likely location for expected values is the previous, manual version of the process you are automating. If you have an Excel file that implements the process you are automating and outputs an `output` tab -- create a test that tries to replicate the output tab!\n",
    "\n",
    "If there is no manual version of the process and you are automating it from scratch, it's a good idea to build from unit tests to integration tests, checking your work manually along the way before building a final expected output you can be confident in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}