Releases: GeoTIFF/geoblaze
Releases · GeoTIFF/geoblaze
v2.8.0
v2.8.0 (2024-08-05)
💥 Breaking Changes
- geoblaze.sum was reverted to the behavior prior to v2.7.0, so now virtual resampling is turned off by default for geoblaze.sum. You can turn virtual resampling on by passing in a vrm and rescale parameter (see more information here: https://docs.geoblaze.io/#sum). Minimal virtual resampling is still turned on by default for min, max, median, mean, and mode calculations.
🚀 New Feature / Improvement
- None
🐛 Bug Fix
- fixed an issue where Github action was breaking because AWS S3 was returning 503 when downloading test files, which was fixed by adding a sleep statement
🏠 Internal
- added
npm run clean, which removes test files that are downloaded while setting up the repo
📝 Documentation
- Updated documentation of geoblaze.sum to show how to enable virtual resampling, passing in a vrm and rescale parameter
Commits
- updated docs 6c1e99e
- updated docs 24e18ea
- updated documentation baee284
- updated documentation for geoblaze.sum 841ab5e
- updated sum to require opt-in to virtual resampling c2441dd
- added 'pip install --upgrade pip' to setup.sh 0e6211d
- improved logging into tile.sh f3d0f2f
- added clean.sh 0acc681
- add sleep fa1ac24
v2.7.0
v2.7.0 (2024-06-19)
💥 Breaking Changes
- geoblaze.sum now applies automatic virtual resampling by default. When a geometry is smaller the one pixel, this will trigger automatic resampling to get the geometry extent to take up half a pixel. This can lead to potentially confusing results. For example, if a geometry takes up 44% of a pixel and the pixel's value is 100, calling geoblaze.sum could return
25(with the pixel automatically divided into quarters). Troubleshooting or debugging results could be difficult, because the sum result wouldn't appear in the histogram of pixel values for the dataset. Future releases will require users to explicitly ask for virtual resampling to be applied when callinggeoblaze.sum.
🚀 New Feature / Improvement
- Added virtual resampling to
src/intersect-polygon/intersect-polygon.module.js. You can now call intersectPolygon with an optional vrm param (representing "virtual raster multiplier") likeintersectPolygon(georaster, geometry, () => {...}, { vrm: [3,2] }). vrm is by default [1,1] and represents how many times to divide each pixel horizontally and vertically. So vrm=[3,2] will lead to 6 virtual pixels with the original divided 3 times horizontally and 2 time vertically. You can also pass{ vrm: "minimal" }and the pixel will only be divided enough to avoid the "no pixels found" error. Minimal virtual resampling also has a sanity threshold, so it won't divide a pixel more than 10k times. - Added a few options to the geoblaze.stats function, of type like
{ include_meta: Boolean, rescale: Boolean, vrm: [Number, Number] }. (include_meta and rescale are described immediately below)
- include_meta will attach additional metadata to the array returned by stats and should be treated as an internal function currently used for development purposes only and is subject to change. Information currently includes vrm (virtual resampling multiplier), vph (virtual pixel height) and vpw (virtual pixel width).
- rescale is whether we should rescale area-based statistics according to virtual resampling multipliers. For example if you sum up 100 virtual subpixels with a value of 8, but they all come from an original pixel of value 8. The sum should be 8, not 800. Otherwise dividing a pixel increases the sum results. The specific statistics that are rescaled are:
count,histogram,invalid,product,sum, andvalid
🐛 Bug Fix
- Calling any of the following functions
geoblaze.max,geoblaze.mean,geoblaze.median,geoblaze.min,geoblaze.mode,geoblaze.modes, orgeoblaze.rangeshould now return an answer, fixing a previous bug where an error "No Values were found in the given geometry" was thrown when the geometry was smaller than a pixel.
🏠 Internal
- added test commands for individual statistical functions, so calling
npm run test:medianwill only run the median tests. This was mainly done for convenience during development. - updated development dependency flug
- added isBboxObj to utils to detect bounding box in form
{ xmin: Number, ymin: Number, xmax: Number, ymax: Number } - Minimal virtual resampling has been turned on by default when calling
geoblaze.max,geoblaze.mean,geoblaze.median,geoblaze.min,geoblaze.mode,geoblaze.modes,geoblaze.range, andgeoblaze.sum
📝 Documentation
- No changes
Commits
- added documentation for virtual resampling and rescaling 6d9d308
- Merge pull request #228 from GeoTIFF/issue-56-virtual-resampling 032d074
- fix linting issue by removing unused import in src/sum/index.js 9645c69
- add support for simple bbox obj resampling bf45190
- added frequency to expected test result ae55519
- trying to fix weird ci issue with fetching file by using readFileSync instead 4faff44
- changed debug level f53c595
- try to fix ci testing issues 63b1e90
- removed debug logging 2101fa4
- added frequency to test expectation 846cb74
- removed frequency from stats tests c8475fb
- added scaled valid values to stats test 25f5fdb
- updated flug dfb76b6
- removed old code climate links 49371f0
- updated stats to return valid by default 8b5035f
- serve more files in median test 56c9e16
- added virtual resampling test files 4017f1f
- updated flug 9f2d7f1
- updated flug and added 1 sec gap time between tests f601b6b
- increased served requests in intersect-polygon.test.js bcfa92f
- fixed lint issue 70c3297
- cleaned up more virtual resampling code and added resampling support to georasters fully loaded into memory d4a1d39
- refactored intersect-polygon 4fcae79
- added modes to the package.json 5191d49
- WORK IN PROGRESS 5208f9d
v2.6.1
- updated dufour-peyton-intersection 28a7d08
v2.6.0
v2.6.0 (2023-12-17)
💥 Breaking Changes
- None
🚀 New Feature / Improvement
- Increased speed of calculating a single statistic (e.g., geoblaze.min, geoblaze.max, geoblaze.mean, geoblaze.median, and geoblaze.sum). Depending on the nuances of the data source and request, this can lead to more than 2x speedups. We are now calling calc-stats with the "stats" array param, so that calc-stats only calculates exactly the statistic(s) required and nothing more. Previously, calc-stats was still calculating additional default statistics. The previous way wasn't an issue when calc-stats was initially adopted, but as calc-stats added more statistics, it led to a decrease in performance.
- Added geoblaze.range, calculating the difference between the min and max value for each band.
🐛 Bug Fix
- None
🏠 Internal
- Created geoblaze.stat for calculating a single statistic. geoblaze.sum and others were refactored to use this. But geoblaze.stat is not a part of the public API.
- Created a separate geoblaze.stat function inside of the lite folder and refactored the function calls in the lite folder.
- Replaced utils.resolve with Quick-Promise
- Added lite folder to format script
📝 Documentation
- No changes
Commits
- Merge pull request #227 from GeoTIFF/refactor-stats bfd42ae
- refactored lite 6c35a01
- increased serve limit for range.test.js 6fb06f6
- updated median 3102076
- updated imports into tests ef3e563
- fixed typo in max test 397d6e4
- fixed typo in modes 768f75c
- replaced resolve 09e832c
- Merge branch 'master' of github.com:GeoTIFF/geoblaze into refactor-stats 177e3c7
- refactored stats be8f405
v2.5.0
v2.5.0 (2023-11-29)
💥 Breaking Changes
- New point densification feature mentioned below may result in slightly more accurate results (i.e. including a pixel or two when they were previously excluded). In general, most results will stay the same.
🚀 New Feature / Improvement
- Point Densification: If you pass in a polygonal geometry in a projection different from the raster, geoblaze will automatically drop 3 additional vertices on each line segment when reprojecting the geometry to the raster projection. This will lead to more accurate reprojection, especially when reprojection leads to a significant amount of warping.
🐛 Bug Fix
- fixed #224, which was an issue where geoblaze would sample the same pixels multiple times when a multipolygon geometry crosses the antimeridian and the same pixel intersects multiple polygons.
🏠 Internal
- updated reproject-geojson to 0.5.0 to support point densification
- added geoaffine
- added clip-test-clipped.json and fiji_anticross_random_test.tif to data/antimeridian for testing
- added new test to stats.test.js to test for multipolygons that cross the antimeridian
📝 Documentation
- No changes
commits
v2.4.0
v2.4.0 (2023-10-13)
💥 Breaking Changes
- for geometries that are non-rectangular polygons or multi-polygons, refactored stats calculation such that no-data values are now included in the total
"count"property of the return object. Previously, geoblaze completely ignored no data values. However, this shouldn't impact any of the statistical results like sum and median. If you want to get the total number of valid values in an area, look for the"valid"property value in the stats result object. Note: this builds on the work in v2.3.0, which accidentally only included this no-data reporting for single rectangles
🚀 New Feature / Improvement
- see breaking change mentioned above
🐛 Bug Fix
- see breaking change mentioned above
🏠 Internal
- The version of GADM that we use for geoblaze testing doesn't appear to be hosted by UC Davis anymore, so I am hosting it on the geoblaze S3 bucket, so we don't lose it
- unzip and use MapSPAM global wheat raster for testing
- update tests to account for new improved logic around counting no-data values
- pass along debug_level to intersect-polygon
- wrote test for issue #212 , proving that it now works and can be closed. There was previously some discrepancy in combined results if you pass in polygons separately or as a multi-polygon
📝 Documentation
- No changes
commits
- Merge pull request #221 from GeoTIFF/issue-212 3f24625
- updated expected results for stats test bc4589c
- updated stats 0dfa38e
- updated intersect-polygon test to account for changes f4dcb12
- updated gadm download link 5eca869
- removed unused noDataValue declaration in intersect-polygon 537a6c5
- report no data values for polygons c9b54e7
- unzip mapspam wheat geotiff on setup 2b41bad
- Merge pull request #220 from GeoTIFF/geojson-test-data db09e2f
- fixed typo in eez test 627a233
- Merge branch 'master' of github.com:GeoTIFF/geoblaze into geojson-test-data 2e594be
- added code to use geojson-test-data 30336f8
- Merge pull request #219 from GeoTIFF/new-test-new-zealand-eez 7689c07
- added test case for New Zealand Land+EEZ intersecting habitat 6e3bd32
- Update CHANGELOG.md 7e4ce2d
v2.3.0
v2.3.0 (2023-09-17)
💥 Breaking Changes
- refactored stats calculation such that no data values are now included in the total
"count"property of the return object. Previously, geoblaze completely ignored no data values. However, this shouldn't impact any of the statistical results like sum and median. If you want to get the total number of valid values in an area, look for the"valid"property value in the stats result object. - improved antimeridian support (described below) could potentially lead to changed results for geometries that cross the antimeridian. The new results should be treated as more accurate.
🚀 New Feature / Improvement
- when a multi-polygon is used where the polygons are far apart, geoblaze will now make multiple smaller samples instead of sampling the whole area that contains all the polygons. A practical example is when a multipolygon includes polygons that border each side of the antimeridian, previously GeoBlaze would sample a bounding box across the whole world (from nearly -180 to 180 longitude). This could lead to memory issues and stalling. Now, GeoBlaze samples from each side of the world separately.
🐛 Bug Fix
- fixed seemingly never-ending stalling when sampling across the antimeridian
🏠 Internal
- refactored stats to use the defaults as defined by https://github.com/danieljdufour/calc-stats
- added a geojson and shapefile that cross the antimeridian to the test data
- replace quick-resolve with quick-promise
📝 Documentation
- No changes
commits
- Merge pull request #217 from GeoTIFF/antimeridian-2023-08-24 836ad0f
- added across.dbf and across.shx 6c944cc
- added data/antimeridian/across.{prj,shp} 8519fa6
- sample twice when crossing antimeridian 19b1ff7
- added more truth data 344d4c4
- added stalling anti-meridian test f7ec999
- Merge pull request #216 from GeoTIFF/DanielJDufour-patch-1 dd2cbec
- Update CHANGELOG.md fc9279b
v2.2.0
v2.2.0 (2023-08-12)
💥 Breaking Changes
- none
🚀 New Feature / Improvement
- stats now includes product (all pixel values within an area multiplied together) because we upgraded calc-stats to the newest version
🐛 Bug Fix
- intersect-polygon now accounts for cases where the geometry's extent is totally outside the extent of the geotiff, returning early
- upgrading snap-bbox, fixing where pixels on the edge of a geotiff where missed
🏠 Internal
- added some tests
- updated some test values to account for new product value returned by calc-stats
- added some more test data
📝 Documentation
- No changes
commits
- Merge pull request #215 from GeoTIFF/updates-2023-08-11 c719d44
- added product to stats tests after updating calc-stats 585c3a7
- updated cross-fetch, georaster, and mathjs dcd58df
- updated calc-stats 7529a41
- increased number of serves in intersect polygon test 28de1d2
- fixed edge case 385422b
- updated snap-bbox 49e70e2
- updated dufour-peyton-intersection 48e9b24
- Update CHANGELOG.md a9fc282
v2.1.0
v2.1.0 (2023-07-23)
💥 Breaking Changes
- Updated convertCrsBboxToImageBbox to properly exclude pixels that barely touch a polygon (i.e. whose pixel centroid doesn't intersect the aoi geometry). This will mean that stats results for rectangular aois could change. You might also see an error thrown saying no valid pixels, whereas before some intersecting pixels were mistakenly included in the calculation. Changes are only expected for rectangular geometries whether they are defined as bounding boxes or rectangular polygons (geoblaze is smart enough to identify a bbox masquerading as a polygon). Non-rectangular geometries should not be affected by any of these changes.
🚀 New Feature / Improvement
- Refactored convertCrsBboxToImageBbox in utils.module.js, so that only pixels whose centroid is within the provided rectangle are used for stats calculations. This is making sure rectangles and bounding boxes are treated the same as other polygons.
🐛 Bug Fix
- Fixed bug in our Github CI workflow whereby gdal_translate wasn't working, because I had forgotten to add instructions to install it
🏠 Internal
- added roundDown function to utils.module.js (copying it from dufour-peyton-intersection)
- added more testing
- committed some new test data files
📝 Documentation
- No changes
Commits
- Merge pull request #214 from GeoTIFF/rect-2023-07-22 44a3987
- updated test value 542b7fd
- updated bash script to exit on any error 4444890
- added gdal to github actions d6f4de4
- expected EXPECTED_BBOX_STATS 3c15b36
- updated median test value 028bd52
- updated median test value a090cec
- updated test value 6f9c7be
- added antimeridian test file 26e1a8e
- debugging ci aa9a26e
- debugging fetching of geojson 87c0739
- replace cross-fetch/node-fetch with built-in fetch for intersect-polygon.test.js 83a2da1
- updated expected test data for a histogram test a201e37
- removed extra console.dir 33e5622
- fixed issues whereby calculating stats in a rectangle would pull all pixels that touch the rectangle and not just those whose centroid is inside e6599d3
- Create CHANGELOG.md afa32b9
v2.0.0
v2.0.0 (2023-07-10)
💥 Breaking Changes
- Pixels whose centroid lies precisely on the intersection of the polygon line segment and the center-line of a pixel row are now snapped out towards the exterior of the polygon. (In other words, we round down from 0.5 not up when calculating the left edge or minimum x value of a strip of intersecting pixels)
🚀 New Feature / Improvement
- Rigorous support for topologically "complex" geometries with overlapping holes or polygons. There is a small performance hit for geometries with holes or multiple polygons because of the new deconfliction/merging steps.
🐛 Bug Fix
- Fixed bug where the bounding box calculation for MultiPolygon geometries was sometimes only using the first polygon.
- Fixed bug whereby overlapping polygons or overlapping holes were misidentifying overlap as "inside" the polygon
- Fixed bug where sometimes No Data or NaN values were passed to the statistical calculations
🏠 Internal
- Replaced bespoke code for pulling polygons from a provided geometry with mpoly
- Replaced bespoke bounding box calculation with bbox-fns
- Used rasterstats to generate "truth" test data. Results are now very similar to rasterstats, often identical. However, sometimes rasterstats and geoblaze will have results that are a few pixels different (representing less than 1% of pixels) for large complex areas. Further investigation is required.
- Use write-image internally to visualize intersections, adding a manual visual way to check intersection calculations
- Use new version of dufour-peyton-intersection
📝 Documentation
- No changes
commits
- removed unecessary resampleMethod in get.module.js e80dc0a
- Merge pull request #206 from GeoTIFF/update-2023-06-10 df1b8a3
- force babel conversion of mpoly and geowarp 18ae9de
- fixed eslint issue 5f3438b
- improved support for multipolygons and polygons with holes 626041e
- added expected_data.txt 3ff5cc4
- updated data/create_expected_truth_data.py dbee4bb
- removed coupling 93e5d3a
- removed mergeRanges b9f0240
- remove categorizeIntersection 242dfb7
- removed cluster and getBoundingBox references 04c96fa
- deleted unused clusterLineSegments (it had moved to dufour-peyton-intersection), and deleted one extaneous getBoundingBox test 261a856
- don't pass no data values to calcstats 049fee0
- cleaned up a little code in get.module.js fab69e3
- added code to run python 3bc2480
- updated ci to use python eb78519
- added geojson test files 21a87d6
- added create_expected_truth_data.py e17eb47
- Merge pull request #205 from GeoTIFF/calc-bbox f9190a6
- replaced bespoke internal logic of getBoundingBox with bbox-fns/calc.js 9ed5ed1
- updated deps and add @turf/boolean-clockwise, bbox-fns, @danieljdufour/write-png, and write-image c4fcf15