Iterations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Interations in Python\n",
    "The inspiration for this section of the talk comes from and borrows heavily upon Ned Batchelder's PyCon 2013 talk titled _\"Loop like a native: while, for, iterators, generators\"_.\n",
    "\n",
    "Please see https://nedbatchelder.com/text/iter.html for the original material which is a lot more extensive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Looping over a list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "cats = ['Max', 'Chloe', 'Bella', 'Oliver', 'Kitty Purry']\n",
    "cats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### First attempt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(len(cats)):\n",
    "    print(cats[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### Second attempt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat in cats:\n",
    "    print(cat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Looping over a dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cats = {'Oliver': 5000, 'Max': 20000, 'Chloe': 15000, 'Bella': 10000, 'Kitty Purry': 1}\n",
    "# These are apparently real cat names, and the first 4 are the most common ones in the USA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat_name in cats:\n",
    "    print(cat_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "So this is actually just looping over looping over keys.\n",
    "How do we get the counts?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat_name in cats:\n",
    "    print(cats[cat_name])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "That worked, but it's not very elegant or readable. We can do better."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat_count in cats.values():\n",
    "    print(cat_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "In reality though, that doesn't seem very useful. Having counts like that is probably not all that useful.\n",
    "This is especially given the order of a dictionary cannot be relied upon."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat_name in cats:\n",
    "    print(cat_name, cats[cat_name])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "That works, but again it's not very expressive. Python gives dictionaries a .items() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat_tuple in cats.items():\n",
    "    print(cat_tuple[0], cat_tuple[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "That's a bit better, as we're not repeating cats, but we can actually make it a lot clearer by using tuple unpacking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example of a tuple\n",
    "list(cats.items())[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Unpacking the tuple\n",
    "for cat_name, cat_count in cats.items():\n",
    "    print(cat_name, cat_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "###### .keys() - what is it for?\n",
    "\n",
    "PEP20 talks about the Python philosophy and is a good guide for what one should strive to achieve when writing Python code and among other things it says:\n",
    "\n",
    "> __There should be one-- and preferably only one --obvious way to do it.__\n",
    "\n",
    "> Although that way may not be obvious at first unless you're Dutch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat in cats.keys():\n",
    "    print(cat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(cats.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "It's actually to be used with set operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# cats - {'Kitty Purry'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cats.keys() - {'Kitty Purry'}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So .keys() exist as to return what is in effect a set that can be used for normal set operations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center>![Kitty Purry](http://i.perezhilton.com/wp-content/uploads/2012/01/katy-perrys-kitty-purrah__oPt.jpg \"Kitty Purry\")</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## List comprehension\n",
    "Let's say we want to store cat names as upper case"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's get a cat names list from the dictionary keys\n",
    "cat_names = list(cats.keys())  # You don't really need list() because of duck typing\n",
    "cat_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat_names_upper = [cat_name.upper() for cat_name in cat_names]\n",
    "cat_names_upper"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Dictionary comprehension\n",
    "Let's say we want to store cat names as upper case and we want to increment the number of cats by one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_cats = {cat_name.upper(): cat_count + 1 for cat_name, cat_count in cats.items()}\n",
    "new_cats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What if we only want actually common cat names?\n",
    "common_cats = {cat_name: cat_count for cat_name, cat_count in cats.items() if cat_count > 1000}\n",
    "common_cats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Enumerate\n",
    "What if we want to know on which loop iteration we're currently on?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### First attempt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We could write\n",
    "i = 0\n",
    "for i in range(len(cat_names)):\n",
    "    print(\"Cat name: {}. Order in dictionary: {}\".format(cat_names[i], i))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### Second attempt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "i = 0\n",
    "for cat in cat_names:\n",
    "    print(\"Cat name: {}. Order in dictionary: {}\".format(cat, i))\n",
    "    i += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### Third time's the charm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# But then we have this straggling i that's still around after the loop, we have to increment it on its own statement etc.\n",
    "for i, cat in enumerate(cats):\n",
    "    print(\"Cat name: {}. Order in dictionary: {}\".format(cat, i))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(enumerate(cats))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This works for lines in files and any kind of iterable items, not just strings or integers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Zip\n",
    "What if we have two lists? That's what we have zip for!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's pretend we have to lists, and create them from the previous stuff\n",
    "cats_names = list(cats.keys())\n",
    "cats_counts = list(cats.values())\n",
    "print(cats_names)\n",
    "print(cats_counts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat_name, cat_count in zip(cats_names, cats_counts):\n",
    "    print(cat_name, cat_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Sorting two lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sorted_cat_names, sorted_cat_counts = zip(*sorted(zip(cats_names, cats_counts), key=lambda x: x[1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cat_name, cat_count in zip(sorted_cat_names, sorted_cat_counts):\n",
    "    print(cat_name, cat_count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# What if we want it to be descending rather than ascending?\n",
    "sorted_cat_names, sorted_cat_counts = zip(*sorted(zip(cats_names, cats_counts), key=lambda x: x[1], reverse=True))\n",
    "for cat_name, cat_count in zip(sorted_cat_names, sorted_cat_counts):\n",
    "    print(cat_name, cat_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### i and i+1 at the same time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "i = 0\n",
    "for i in range(len(sorted_cat_names) - 1):\n",
    "    print(\"Current: {}. Next: {}\".format(sorted_cat_names[i], sorted_cat_names[i+1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "for current_cat, next_cat in zip(sorted_cat_names, sorted_cat_names[1:]):\n",
    "    print(\"Current: {}. Next: {}\".format(current_cat, next_cat))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Finding min/max in an interable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Highest count of cats\n",
    "print(\"Max:\", max(cats.values()))\n",
    "\n",
    "print(\"Min:\", min(cats.items(), key=lambda x: x[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Itertools\n",
    "Itertools is a part of the standard library that has a lot of functions to doing interesting custom iterations. These are just some basic examples, just be aware it's there."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "import string\n",
    "repeat_sequence = itertools.cycle(string.ascii_lowercase)\n",
    "for number, _ in zip(repeat_sequence, range(50)):\n",
    "    print(number, end=\" \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat_sequence = itertools.cycle(cats)\n",
    "for cat, _ in zip(repeat_sequence, range(50)):\n",
    "    print(cat, end=\" \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "for cat in itertools.repeat(cat_names[0], 3):\n",
    "    print(cat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Creating your own stream"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import a csv we'll use for the next part\n",
    "import csv\n",
    "def csv_generator(csv_reader):\n",
    "    for row in csv_reader:\n",
    "        for cell in row:\n",
    "            yield cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('sources/tmdb_5000_movies.csv') as f:\n",
    "    csv_reader = csv.reader(f)\n",
    "    for cell, _ in zip(csv_generator(csv_reader), range(20)):\n",
    "        print(cell, end=\" \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def odd_numbers(stream_of_numbers):\n",
    "    for number in stream_of_numbers:\n",
    "        if number % 2 == 1:\n",
    "            yield number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for number in odd_numbers(range(50)):\n",
    "    print(number, end=\" \")"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}