Example for unchipped images

[kvenkman]

Hi, I'm working with a dataset that consists multiple bands (9) of a single area (8000x16000) from which I'd like to create stacks of smaller chips to train a segmentation model. I'm defining my train, validation and test areas as bounding boxes within this single area.

Is there a good way to this, or is there an example dataset/dataloader that I can refer to? I'm currently referring to this discussion, but was wondering if one of the pre-existing datasets tackled something similar. Thanks!

[adamjstewart]

What kind of images and masks are you using? I would suggest creating a new RasterDataset for both the image and mask datasets (if one doesn't already exist), then combine them using an IntersectionDataset (the & character). See https://torchgeo.readthedocs.io/en/stable/tutorials/custom_raster_dataset.html for instructions on how to create a custom RasterDataset. Once you have your intersection dataset, split it into train/val/test using the tools listed in the discussion you link to so you have 3 datasets. Alternatively, you can define 3 samplers with different ROIs and 1 dataset. Let me know if you hit any issues, I'm happy to debug code!

[kvenkman]

Hi @adamjstewart ! Thanks for the response - here's a basic code snippet that I've written.

from torchgeo.datasets import RasterDataset
from torchgeo.datasets.utils import BoundingBox
from torchgeo.samplers import RandomBatchGeoSampler

class ModelInputs(RasterDataset):
    is_image = True
    separate_files = True
    
class ModelLabels(RasterDataset):
    is_image = False

inputdataset = ModelInputs('./input_rasters')
labeldataset = ModelLabels('./labels')

dataset = inputdataset & labeldataset

xmin, xmax, ymin, ymax, tmin, tmax = dataset.bounds

y_extent = ymax-ymin
x_extent = xmax-xmin

train_roi = BoundingBox(xmin, xmax, ymin, ymin+0.8*y_extent, tmin, tmax)
test_roi = BoundingBox(xmin, xmin+0.5*x_extent, ymin+0.8*y_extent, ymax, tmin, tmax) 
val_roi = BoundingBox(xmin+0.5*x_extent, xmax, ymin+0.8*y_extent, ymax, tmin, tmax) 

CHIP_SIZE=256
BATCH_SIZE=16

train_sampler = RandomBatchGeoSampler(dataset, batch_size=BATCH_SIZE, size=CHIP_SIZE, roi=train_roi)
test_sampler = RandomBatchGeoSampler(dataset, batch_size=BATCH_SIZE, size=CHIP_SIZE, roi=test_roi)
validation_sampler = RandomBatchGeoSampler(dataset, batch_size=BATCH_SIZE, size=CHIP_SIZE, roi=val_roi)

Ok, I think I have this working - is the bare bones definition of the ModelInputs and ModelLabels classes sufficient? Related to the ModelIputs class, is there any way I can specify the order in which the layers are ordered?

Lastly (sorry for all the questions!) I'm interested in experimenting with withholding certain input layers to see how it impacts model training - is there a suggested way of doing this?

[adamjstewart]

is the bare bones definition of the ModelInputs and ModelLabels classes sufficient?

This depends on what you're trying to do. At the moment, without any glob or regex, this will match all raster files recursively in a directory. Also, without a regex or datetime format, time is ignored and no time-series support will be provided. If you're fine with these conditions, your definitions are probably sufficient.

Related to the ModelIputs class, is there any way I can specify the order in which the layers are ordered?

Lastly (sorry for all the questions!) I'm interested in experimenting with withholding certain input layers to see how it impacts model training - is there a suggested way of doing this?

It's actually easy to do both of these at the same time. For example, using code like:

class ModelInputs(RasterDataset):
    is_image = True
    separate_files = True

    all_bands = ['B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'B8', 'B9']

If the user uses:

inputdataset = ModelInputs('./input_rasters')

all bands will be used in order. If the user uses:

inputdataset = ModelInputs('./input_rasters', bands=['B3', 'B2', 'B1'])

it will only use the first 3 bands, in reverse order.

RasterDataset has a ton of other cool features, but you may need to read the source code to see all of them. The documentation can only go so far without becoming unreadable. Also, feel free to open a PR to add anything you think is missing from the docs!

[kvenkman]

Yep, the dataset that I'm working with currently does not have a time component to it (or datetime included in the filename string) so I'm ok with excluding that functionality.

Is there any way to verify that the bands are getting loaded in the order that I specify? For example, if I do this:

inputdataset = ModelInputs('./input_rasters', bands=['B3', 'B2', 'B1'])

followed by

print(inputdataset.files),

I still get:

['./final_B01.tif', './final_B02.tif', './final_B03.tif', './final_B04.tif', './final_B05.tif', './final_B06.tif']

I think this behavior is coming from torch.Dataset, but let me know what you think. I'd be happy to modify this behavior and create a PR if it makes sense. Thanks for your help with this!

[adamjstewart]

The files attribute is TorchGeo-specific and doesn't exist in native PyTorch. It doesn't give the order of bands, that is given by inputdataset.bands. If you want to verify that the band order is being loaded correctly, you can try 1, 2, 3 and 3, 2, 1 and print the resulting images from __getitem__ and verify that the order was reversed.

[kvenkman]

Hey @adamjstewart - I wanted to make a suggestion about adding a documentation note about regex in the class definition for Rasterdataset. Do you think this would be useful? I was having issues loading the bands of my data correctly, and I tracked it down to my regex expression being incorrect.

[adamjstewart]

Depends on what you want to add. It's already quite well documented a few lines below and in our Custom Raster Datasets tutorial.