Skip to content

HaydenKaufmann/McMaster_Scraper

Repository files navigation

==========================================
McMaster-Carr Part Scraping and CAD Downloader
==========================================

This project contains two Python scripts designed to automate the process of extracting part numbers from a text file, fetching product details from McMaster-Carr, and downloading CAD files for each part.

-----------------------------------------
Requirements:
-----------------------------------------
- Python 3.x
- Required Python libraries:
  - `selenium`
  - `pandas`
  - `webdriver_manager`

-----------------------------------------
Setup Instructions:
-----------------------------------------
1. Paste McMasterCarr information into a .txt file in the project directory (similar to 'ExampleNutList.txt').
2. Change file names in 'scrapePartNumbers.py'
3. Run `scrapePartNumbers.py` to generate `PartNumbers.csv`.
4. Run `fetchCad.py` to scrape product details and download CAD files.
5. Scraped data will be saved in the folder `part_files`, under subfolders for each part number.

-----------------------------------------
Output:
-----------------------------------------
1. **PartNumbers.csv**: A CSV file containing the extracted part numbers.
2. **Part-specific folders**: Contain the product details and CAD files for each part number.

-----------------------------------------
1. Part Number Extraction Script (scrapePartNumbers.py)
-----------------------------------------
Purpose: 
- Extracts part numbers from a text file (ExampleNutList.txt) and saves them in a CSV file (PartNumbers.csv).

How it works:
1. Reads part numbers from `ExampleNutList.txt`.
2. Uses a regular expression to extract part numbers (e.g., "95462A029").
3. Saves the extracted part numbers into `PartNumbers.csv`.

Usage:
- Place the part numbers in the text file (`ExampleNutList.txt`).
- Run the script to extract part numbers and save them into `PartNumbers.csv`.

-----------------------------------------
2. Product and CAD Scraping Script (fetchCad.py)
-----------------------------------------
Purpose: 
- Automates web scraping of product details and downloading CAD files from McMaster-Carr for each part number listed in `PartNumbers.csv`.

How it works:
1. Reads part numbers from `PartNumbers.csv`.
2. Visits the McMaster-Carr website to retrieve product details (name and specifications) for each part.
3. Saves the product information in a text file and downloads the CAD file for each part (if available).
4. Stores the scraped data and CAD file in a folder corresponding to each part number.

Usage:
- Ensure you have a valid `PartNumbers.csv` file generated by the first script.
- Run the script to scrape product details and download CAD files.


About

Scrapes McMaster-Carr CAD files and product details

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages