How to train a model with GrayScale images?

[bluenight12]

    def load_model(self):
        self.model = Patchcore(
            backbone="wide_resnet50_2",
            pre_trained=True,
            coreset_sampling_ratio=0.1,
            layers=["layer2", "layer3"],
        )

i want to train with grayscale image this code. There's no problem with training. But if i input grayscale image, occurs error.

Failed to set tensor. Check 'is_dynamic || port.get_shape() == tensor->get_shape()' failed at src/inference/src/dev/isync_infer_request.cpp:286:
The input tensor size is not equal to the model input type: got [100,60] expecting [1,3,100,60].

this is my error code

i want to change model input size(channel). how can i do?

[alexriedel1]

Please show the full training code so that we can replicate the bug

[bluenight12]

class TrainMode():
    def __init__(self):
        self.imgsz = (100, 60)

    def load_datamodule(self):
        # Create the datamodule
        self.datamodule = Folder(
            name='example',
            root="datasets/0829",
            normal_dir="good",
            normal_test_dir="test_good",
            #abnormal_dir = 'bad',
            test_split_mode=TestSplitMode.NONE,
            val_split_mode=ValSplitMode.FROM_TRAIN,
            image_size= self.imgsz,
            task="classification",
            eval_batch_size = 1,
        )
    
    def load_model(self):
        self.model = Patchcore(
            backbone="wide_resnet50_2",
            pre_trained=True,
            coreset_sampling_ratio=0.1,
            layers=["layer2", "layer3"],
        )

    
    def load_engine(self):
        self.engine = Engine(
            normalization=NormalizationMethod.MIN_MAX,
            task=TaskType.CLASSIFICATION,    
            accelerator="cpu",
            default_root_dir="./results/logs",    
            deterministic=True,
        )
    
    def export_model(self, path):
        self.engine.export(model=self.model, 
            input_size=self.imgsz,
            export_type=ExportType.ONNX, 
            #compression_type=CompressionType.INT8,
            export_root=os.path.join(os.getcwd(), path),
            datamodule=self.datamodule, 
            ckpt_path='./results/logs/Patchcore/example/latest/weights/lightning/model.ckpt',
        )
    
    def run(self):
        self.load_datamodule()
        self.load_model()
        self.load_engine()
        self.engine.train(model=self.model, datamodule=self.datamodule)
        self.export_model(args.path)

if __name__ == "__main__":
    detector = TrainMode()
    detector.run()

i use this code convert color dataset to grayscale

for root, dirs, files in os.walk(directory):
        for file in files:
            if file.endswith('.png'):
                file_path = os.path.join(root, file)
                img = cv2.imread(file_path)
                gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
                new_dir = root + '_gray'
                os.makedirs(new_dir, exist_ok=True)
                new_file_path = os.path.join(new_dir, file)
                cv2.imwrite(new_file_path, gray_img)
                print(f"Converted {file_path} to grayscale and saved as {new_file_path}")

Train runs normally, but when the input channel changes back to 3 and the Train data is re-entered for inference, the error mentioned above occurs.

and, this is my inference code

self.onx = OpenVINOInferencer(os.path.join(path, 'model.onnx'), task=TaskType.CLASSIFICATION)
img = cv2.imread('img.png', cv2.IMREAD_GRAYSCALE)
pred_score = self.onx.predict(img, self.metadata).pred_score

[alexriedel1]

You are mixing the anomalib image reading based on Pillow with cv2 grayscale conversion and with cv2 grayscale loading. This results in more problems than you are currently aware of. The way to go would be to use an image transform on your dataset for grayscaling your images.

Here you can read how to read image transforms in anomalib https://anomalib.readthedocs.io/en/v1.1.1/markdown/guides/how_to/data/transforms.html

What you will need is this transform: https://pytorch.org/vision/main/generated/torchvision.transforms.v2.Grayscale.html

[bluenight12]

As far as I know, when doing onnxinference, you use numpy, not tensor. You can see this in the following code.

 # Convert file path or string to image if necessary
        if isinstance(image, str | Path):
            image = Image.open(image)

        # Convert PIL image to numpy array
        if isinstance(image, Image.Image):
            image = np.array(image, dtype=np.float32)
        if not isinstance(image, np.ndarray):
            msg = f"Input image must be a numpy array or a path to an image. Got {type(image)}"
            raise TypeError(msg)

this is openvino_inferencer.py

and,

this is my onnx structure. I want to change the channel among these 4(1, 3, 100, 60)

I would like to know how to change the input channel when learning in anomalib.

[samet-akcay]

You could use GrayScale transform from torchvision as part of your transforms. The advantage of this is that Anomalib will use this transform for your inference model, so you don't need to manually apply this transform before forward-passing in your inference.

For more details, you could refer to the custom transforms documentation here.
https://anomalib.readthedocs.io/en/v1.1.1/markdown/guides/how_to/data/transforms.html

[bluenight12]

I have a grayscale image. However, even after applying the changes, the ONNX input size is still 1x3xWxH. How should I handle this?

self.train_transform = Compose(
            [
                Resize(self.imgsz, interpolation=InterpolationMode.BILINEAR, antialias=True),
                Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
                Grayscale(num_output_channels=1), 
            ],
        )

line 456, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[32, 1, 100, 60] to have 3 channels, but got 1 channels instead

[blaz-r]

I believe that the feature extractor requires 3 channels. I'm not sure what would be the best way to fix this, but I'd say that changing the grayscale to be 3 channel grayscale would probably work.