You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Calculating box checksums on demand has too high performance cost for http request (~1.5GB box takes about 5s to calculate md5 checksum). So we need to calculate checksums on background, persist results and on request just read those checksums.
checksum calculation must not block any request -> must run on background
BackgroundChecksumCalculator calculates checksums with given configuration for all boxes and persist them with HashStore
BackgroundChecksumCalculator must keep checksums in latest state -> if new box is added when Boxitory already running, it must catch it and calculate checksum
once checksum is calculated and persisted, it can't be replaced
new configuration option box.checksum_precalculate
boolean value - true|false
tells whether checksums for boxes should be precalculated on background
default true
probably there will be some other (advanced) configuration option configuring background calculation
probably BackgroundChecksumCalculator interface with FilesystemBackgroundChecksumCalculator implementation
Don't forget to test and keep it as independent as possible. Again, there must be space for replace storing checksums to file by e.g. store in database.
The text was updated successfully, but these errors were encountered:
Calculating box checksums on demand has too high performance cost for http request (~1.5GB box takes about 5s to calculate md5 checksum). So we need to calculate checksums on background, persist results and on request just read those checksums.
BackgroundChecksumCalculator
calculates checksums with given configuration for all boxes and persist them withHashStore
BackgroundChecksumCalculator
must keep checksums in latest state -> if new box is added when Boxitory already running, it must catch it and calculate checksumbox.checksum_precalculate
true|false
true
BackgroundChecksumCalculator
interface withFilesystemBackgroundChecksumCalculator
implementationThe text was updated successfully, but these errors were encountered: