Issue in SSIM implementation

[ahmadmughees]

Hey, you have amazing work. I am facing an issue with SSIM score you have calculated. it's like reshaping the image and passing it from SSIM function is not consistent with the Original Implementation of authors of SSIM. See the code for reproducing please.

% LR image available here 'https://raw.githubusercontent.com/mugheesahmad/Fun_testing/master/LR0000001.jpg' 
% HR image available here 'https://raw.githubusercontent.com/mugheesahmad/Fun_testing/master/HR0000001.jpg' 
lr = imread('LR0000001.jpg');
hr = imread('HR0000001.jpg');
ssim(hr, lr) %colored image
% ans = 0.8433
ssim(rgb2gray(hr), rgb2gray(lr)) %builtin MATLAB function
% ans =    0.7570
original_ssim(rgb2gray(hr), rgb2gray(lr)) %author implementation available here https://ece.uwaterloo.ca/~z70wang/research/ssim/
% original implementation doesnot accept the RGB image
% ans =    0.7574 
lru = reshape(lr, [380*672,3]);  %your way of doing
hru = reshape(hr, [380*672,3]);
ssim(lru, hru)
%ans = 86.70

As per the docs of Matlab SSIM, only gray images can be passed. Your way of using it does not consistent with the original and also with the SSIM implementation with skimage and pytorch version. see this and this colab file.

[Coldog2333]

Hello, thank you for your bug report!
Actually, the evaluation implementation was copied from the repository of the authors: https://github.com/anchen1011/toflow
You can compare the Matlab codes of ours and find out the reason.
It is possible that they corrected the implementation after I implemented this torch-version.

Thank you again and I will check it later. :)

[ahmadmughees]

Hi, thanks for your response. No, they didn't update any code. I also posted the issue at their repository but haven't heard from him yet. With this bug, all the results posted are not consistent.

Thanks alot.