Image-Warp-PyTorch

This is a customized PyTorch operation for replacement of nn.functional.grid_sample(). Warping (a.k.a. reprojecting) is an essential step in Temporal Anti-aliasing, Real-time Path Tracing Denoising, etc. It can morph the image according to the motion vectors (a.k.a. optical flows) to align it with another image.

Usage

image_warp.apply(image, motion_vector)

Warp the input image with size {N, C, H, W} according to the motion vector with size {N, H, W, 2}.

image_warp_accum.apply(image_src, image_dst, motion_vector, alpha=0.5)

Warp the image_src and blend it with the image_dst. The motion_vector maps the target image to the source image.

Letting I_src denote image_src, I_dst denote image_dst, α denote alpha, v denote motion_vector, and I_out denote returning value, this function can be defined as:

Compile

bash ./install_kernel_filter.sh

Difference from torch.nn.functional.grid_sample()

The input motion_vector stores coordinates in a relative manner. E.g., a motion vector of [-5, 8] will fill the I_dst(x, y) with the value of I_src(x-5, y+8). On the other hand, the input grid of torch.nn.functional.grid_sample() stores coordinates in an absolute manner.

The inference speed is faster than torch.nn.functional.grid_sample().