Consistency guided multiple plane image construction for novel view synthesis.

Recently, Multi-Plane Image shows great potential in novel view synthesis since it provide a generalizable formulation that enables strong reasoning ability even in unknown scenes. However, existing methods not only struggle with occlusion and complex scenes, but also often require a large number of depth planes and expensive computational cost. In this paper, we propose a novel consistency guided Multi-Plane Image construction method specifically designed for novel view synthesis. Different from the previous MPI-based methods, we construct the MPI serially layer by layer and accumulate consistency information during the process. Specifically we first propose a cross-view consistency mask to incorporate foreground occlusion information into the layered construction to perceive occlusion. Second, we propose a cross-layer consistency mask and a novel depth guidance strategy to incorporate appropriate scene context information into the layered construction to better understand the geometric structure of the scene. We conduct extensive experiments on the Spaces and Real Forward-Facing datasets. The results demonstrate that our method excels in novel view synthesis and multi-frame denoising tasks, achieving state-of-the-art performance with relatively low computational cost. Quantitatively, it outperforms state-of-the-art methods by improving PSNR by approximately 1.1% in challenging sparse-view settings and 2.1% in denoising tasks, achieving superior performance with relatively low computational cost.