scarapingai

Extract data from websites automatically with AI or build web scraping agents for bulk URL scraping.

Installation

Install it via npm:

npm i scarapingai

Highlights

• Built-in residential proxies and captcha handling
• Smart ad blocker, popup blocker for better performance
• Accept cookie consent automatically to close cookie banners
• Compatible with Puppeteer, Playwright for browser automation and testing.
• Background jobs for bulk URL scraping with automatic retry & error handling.

Usage

Get your api key from here

const agenty = new Agenty(API_KEY);
const data = await agenty.browser.extract("https://example.com");
console.log(data);

Extract

To auto-extract product, jobs listing, SEO meta data, schema JSON etc from given URL

const data = await agenty.browser.extract("https://example.com");
console.log(data);

Scrape

To extract data from given CSS selector or custom jQuery function

const data = await agenty.browser.scrape("https://example.com");
console.log(data);

Screenshot

To capture a screenshot for given URL

const data = await agenty.browser.screenshot("https://example.com");
console.log(data);

PDF

To convert webpage into PDF.

const data = await agenty.browser.pdf("https://example.com");
console.log(data);

Content

To get HTML content from a URL.

const data = await agenty.browser.content("https://example.com");
console.log(data);

License

scrapingai is a project by Agenty, released under the MIT License.