README.md

Javascript Pachong

Pachong which means a generic term for vertebrates in chinese, you can find here chinese dictionary, and it's something like crawler.

This is a crawler library written in javascript, so you can use this in server side or browser.

Usage

npm install jspachong

Benchmark

It will crawl 10 pages parallelly and sequentially.

npm run benchmark

Server Side

var Pachong = require('jspachong')
var crawler = new Pachong(requestObject, options)

crawler.queue(requestObject)
       .queue(requestObject)
       .queue(requestObject)
       .run()
       .then((res) => {})
       .catch((err) => {})

• requestObject

  request library options

  simple: {
    method: 'GET',
    uri: 'https://www.google.com'
    callback: function (err, res) {
      if (err) return
      // Do something here...
    }
  }
  

  see request document for more information

• options

  parallel bool
  Run crawlers parallel.
  
  max integer
  Max crawlers run each time.
  

Browser

To do.

License

MIT