This script scrapes a Zillow page and dumps all current listed homes to a file. Optionally, it can also send you an email when new homes are posted.
Clone our repository
git clone https://github.com/alexliu0809/Zillow_Scraper
Install python libraries.
cd Zillow_Scraper
# Optionally: start a virtual environment
python3 -m venv .
source bin/activate
# Mandatory
python3 -m pip install -r requirements.txt
Find a Zillow page that lists homes to scrape and COPY its URL(see below for an example).
The page listed above has this URL:
https://www.zillow.com/san-diego-ca/rentals/2-_beds/?searchQueryState=%7B%22pagination%22%3A%7B%7D%2C%22usersSearchTerm%22%3A%2292037%22%2C%22mapBounds%22%3A%7B%22west%22%3A-117.24590349026118%2C%22east%22%3A-117.1956925375024%2C%22south%22%3A32.85766848585083%2C%22north%22%3A32.88506209326565%7D%2C%22mapZoom%22%3A15%2C%22regionSelection%22%3A%5B%7B%22regionId%22%3A54296%2C%22regionType%22%3A6%7D%5D%2C%22isMapVisible%22%3Atrue%2C%22filterState%22%3A%7B%22beds%22%3A%7B%22min%22%3A2%7D%2C%22fore%22%3A%7B%22value%22%3Afalse%7D%2C%22ah%22%3A%7B%22value%22%3Atrue%7D%2C%22sort%22%3A%7B%22value%22%3A%22days%22%7D%2C%22nc%22%3A%7B%22value%22%3Afalse%7D%2C%22cmsn%22%3A%7B%22value%22%3Afalse%7D%2C%22fsba%22%3A%7B%22value%22%3Afalse%7D%2C%22fr%22%3A%7B%22value%22%3Atrue%7D%2C%22fsbo%22%3A%7B%22value%22%3Afalse%7D%2C%22auc%22%3A%7B%22value%22%3Afalse%7D%7D%2C%22isListVisible%22%3Atrue%7D
In zillow_scraper.py
, change the URL placeholder at line 18 to the real URL copied above:
# Replace "YOUR_ZILLOW_URL" with the URL of the page you are interested in:
ZILLOW_URL = r"YOUR_ZILLOW_URL"
You can send emails to your own email account when a new home is posted. You need to config three things:
- SENDING_GMAIL_ACCOUNT: a gmail account for sending the notification
- SENDING_GMAIL_PASSWORD: the password for the above gmail account. Note that you might need to create an application specific password for this part to work.
- RECEIVING_EMAIL_ACCOUNT: the email account(s) for receiving the notification. You can have multiple receiving accounts here.
The default setup looks like below in zillow_scraper.py
(line 19 - 21):
SENDING_GMAIL_ACCOUNT = "YOUR_GMAIL_ACCOUNT"
SENDING_GMAIL_PASSWORD = "YOUR_GMAIL_PASSWORD"
RECEIVING_EMAIL_ACCOUNT = ['RECEIVING_EMAIL_ACCOUNT',]
You need to replace the placeholders with real email accounts and passwords. An example configuration:
SENDING_GMAIL_ACCOUNT = "alex@gmail.com"
SENDING_GMAIL_PASSWORD = "alexpassword"
RECEIVING_EMAIL_ACCOUNT = ['bob@gmail.com',"charlie@outlook.com"]
One way to run the script (this run the script every 2 hours)
screen -S scrap; # optionally create a screen session
while true; do python3 zillow_scraper.py; sleep 7200; done # Run the script every 7200 seconds
This script is not designed for large-scale scraping. Use it at your own risk (licensed under the Apache License).