Releases: 68publishers/crawler
Releases · 68publishers/crawler
v0.5.6
v0.5.5
0.5.5 - 2023-11-13
Fixed
- Fixed Content-Length calculation method in POST requests to
callbackUri
.
v0.5.4
0.5.4 - 2023-10-27
Added
- Added the scenario option
config.options.proxyUrls
. - Added action
evaluate
. - Added strategy
evaluate
in the actioncollectData
.
v0.5.3
0.5.3 - 2023-10-23
Changed
- Database connections now always have UTC timezone.
v0.5.2
0.5.2 - 2023-10-18
Added
- Added errors logging in the Crawler's
errorHandler
handler.
Fixed
- Fixed the value of the field "finishedAt" in the result returned from Crawler.
v0.5.1
0.5.1 - 2023-09-20
Fixed
- Fixed cookies transferring between requests with different sessions.
v0.5.0
0.5.0 - 2023-07-10
Added
- Added field
active
for scenario schedulers. The field is required for POST/PUT endpoints, and it is also returned by GET endpoints. - Added endpoints
PUT /api/scenario-schedulers/:scenarioSchedulerId/activate
andPUT /api/scenario-schedulers/:scenarioSchedulerId/activate
.
v0.4.3
0.4.3 - 2023-07-10
Changed
- Changed pm2 configuration file for docker images
68publishers/crawler:{version}-app
. The application is launched in thefork
mode instead of thecluster
mode.
Fixed
- Fixed zombie chrome processes - worker container is run with the option
--init
- the process should be used as the PID 1 in the container.
v0.4.2
0.4.2 - 2023-07-03
Added
- Added separated docker images for development (
dev-app
,dev-scheduler
,dev-worker
). - Added separated docker images for production (
prod-app
,prod-scheduler
,prod-worker
). Existing all-in-one image has now targetprod-all
.
Changed
- Log files in
var/log
are now kept for 7 days only. - The crawler waits for the request queue to empty before exiting (until now, it only waited during an aborted scenario).
- Changed Release GitHub Action. Images for specific entrypoints are now also released with pattern
68publishers/crawler:{$version}-{app|scheduler|worker}
.
v0.4.1
0.4.1 - 2023-06-23
Changed
- Developer containers have been resized (chrome is installed for the
worker
image only).