How to ignore parts of depth image?

[thundercarrot]

I'm supplying my own depth image. I also have a mask that indicates where depth image pixel values are invalid. Setting invalid depth image pixels to 0 seems wrong, as that would mean "very far away".

How can I use this mask with the depth image model?

[lllyasviel]

Midas is inverse relative depth and the formulation is here so basically 0 means far away.
I am assuming we are talking about depth map obtained by sensors and it has some holes in it, and we will need some depth map inpainting preprocessing. It seems that people like to use super pixels to inpaint depth maps while I think iterative joint Bilateral filter in the holes should be good enough