You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some Spidermon built-in monitors (like ItemValidationMonitor) increases the size of the stats logged, and because when trying to upload that to Scrapy Cloud, we receive a "Value exceeds max encoded size of 65536 bytes" error message.
One solution suggested was to remove the stats in Spidermon based on a new setting. However, the size limitation is neither Spidermon nor a Scrapy problem, as we don't have limits in how we can use the stats in both libraries.
It is a limitation in a vendor tool (Scrapy Cloud), so one possible solution is to update HubStorageStatsCollector that is the Stats Collector used by Scrapy Cloud to handle this scenario.
The text was updated successfully, but these errors were encountered:
Some Spidermon built-in monitors (like ItemValidationMonitor) increases the size of the stats logged, and because when trying to upload that to Scrapy Cloud, we receive a "Value exceeds max encoded size of 65536 bytes" error message.
One solution suggested was to remove the stats in Spidermon based on a new setting. However, the size limitation is neither Spidermon nor a Scrapy problem, as we don't have limits in how we can use the stats in both libraries.
It is a limitation in a vendor tool (Scrapy Cloud), so one possible solution is to update HubStorageStatsCollector that is the Stats Collector used by Scrapy Cloud to handle this scenario.
The text was updated successfully, but these errors were encountered: