visibility into sub-regions within a UnionDataset

[roybenhayun]

Hi,
consider the following case for a segmentation task:-there is a Sentinel2 Tile and 3 area of interest regions for masking in different areas of the tile. each such masking AOI region is relatively small. if we use a single mask for the three, the bounding box of the IntersectionDatabase would be very big and many samples will be irrelevant for segmentation training.
just for illustration, let's say it looked like:

so we try to use the cool feature of interesction and union of datasets. we create 3 masks, per each AOI and use intersections and unions in this way:

    def setup(self, stage=None):
        self.eo_ds = Sentinel2(root=r'\foo', transforms=sentinel2_preprocess_image)

        self.mask_ds_1 = RasterDataset(r'\subset_1\10m', transforms=preprocess_mask)
        self.mask_ds_1.is_image = False
        self.mask_ds_2 = RasterDataset(r'\subset_2\10m', transforms=preprocess_mask)
        self.mask_ds_2.is_image = False
        self.mask_ds_3 = RasterDataset(r'\subset_3\10m', transforms=preprocess_mask)
        self.mask_ds_3.is_image = False

        ds1 = self.eo_ds & self.mask_ds_1  # Tile & top right mask
        ds2 = self.eo_ds & self.mask_ds_2  # Tile & bottom right mask
        ds3 = self.eo_ds & self.mask_ds_3  # Tile & bottom left mask

        self.ds = ds1 | ds2 | ds3  # union of the 3 masked regions
        print(f"ds: {self.ds}")

the output is:

ds: UnionDataset Dataset
    type: UnionDataset
    bbox: BoundingBox(minx=..., maxx=..., miny=..., maxy=..., mint=..., maxt=...)
    size: 3

first, just to be sure, is this a right way to use intersections and unions with TorchGeo?

the other question is how to get visibility into the sub regions in the resulting UnionDataset? seeing a BoundingBox in the console output is somwhat confusing since intuitively would assume there are 3 bounding boxes per each intersection. would have been good to be able to get some way of inspecting \ debugging the UnionDataset mostly for visibility and validation. perhaps the 3 bounding boxes, or other.

so how are the sub regions represented, can we be sure there are indeed small sub regions and not a big bounding box as we hoped to avoid, and how can we inspect validate the union of sub regions?

thanks!

[adamjstewart]

It's actually even simpler than that. If you have a directory image containing one or more Sentinel-2 images and mask containing 3 or more land cover subsets, you can use:

def setup(self, stage):
    eo_ds = Sentinel2("image")
    mask_ds = RasterDataset("mask")
    self.ds = eo_ds & mask_ds

You'll also want to declare one or more samplers here. TorchGeo will automatically only consider the regions highlighted in red where you have both image and mask data, it will never sample from the rest of the region.

To inspect further, you can access the dataset index attribute. All GeoDatasets in TorchGeo are modeled as R-trees. eo_ds.index is an R-tree containing the spatiotemporal bounding box of all Sentinel-2 tiles. mask_ds.index is the R-tree for the masks. And ds.index is the collection of bounding boxes that intersect between the two.

For more details, see:

• R-tree documentation: will tell you how to iterate over an index or compute an intersection
• torchgeo.datasets.geo source code: where all of these magic base classes are actually defined