git-scraper-example

A example of a git scraper that downloads, commits and archives a data set.

The source for this demonstration is the list of liquor stores in the state of Iowa. The scraping routine is automated by the scrape.yaml workflow.

Instructions

Clone the repository

git clone git@github.com:palewire/git-scraper-example.git

Move into the directory

cd git-scraper-example

Install the Python tools

pipenv install

Run the downloader

pipenv run python src/download.py