Layer-based View Synthesis with Depth Maps
Summary: View Synthesis is an exciting technique in image based rendering (IBR). It includes some critical steps as pre-processing of depth map, 3-d image warping and hole filling etc. In 3-d image warping, the transform consists of reprojection (2D-to-3D) and subsequent backprojection (3D-to-2D). In new view generation, two challenging problems should be handled, i.e. visibility (folds) and de-occlusions (holes) . Here the visibility is verified by Z-buffer rather than a scanning order arrangement, and "holes" are filled by a layered-based rendering scheme, where each reference view is represented by a main layer, a boundary layer and a background layer. (Note: The testing data is multiple view videos "Breakdancer" and "Ballet" from Microsoft Research Lab, with known calibration parameters and depth maps.)
------------------------------------------------------------------------------------------------------------------
View Synthesis with One View plus Depth Map
-----------------------------------------------------------------------------------------------------------------
Reference view
Depth image
Synthesized view without visibility verification and hole filling
Synthesized view without hole filling
Synthesized view with both visibility verification and hole filling
Free Viewpoint Rendering (Rotation)
---------------------------------------------------------------------------------
View Interpolation with Two Views plus Depth Maps
---------------------------------------------------------------------------------
GUI for source view/depth play
GUI for view synthesis without hole filling (topright: error)
GUI for view synthesis with hole filling (topright: error)
----------------------------------------------------------------------------------------------------------------- 
Use the main layer only (view 1)
Apply the boundary and background layer (view 1)
Use the main layer only (view 3)
Apply the boundary and background layer (view 3)
Use the main layer only (view 6)
Apply the boundary and background layer (view 6)