Reconstructing Occluded Surfaces using Synthetic Apertures:
Stereo, Focus and Robust Measures


Vaibhav Vaish

Stanford University

Richard Szeliski

Microsoft Research

C. L. Zitnick

Microsoft Research

Sing Bing Kang

Microsoft Research

Marc Levoy

Stanford University


To appear in CVPR 2006




Most algorithms for 3D reconstruction from images use cost functions based on SSD, which assume that the surfaces being reconstructed are visible to all cameras. This makes it difficult to reconstruct objects which are partially occluded. Recently, researchers working with large camera arrays have shown it is possible to ``see through" occlusions using a technique called synthetic aperture focusing. This suggests that we can design alternative cost functions that are robust to occlusions using synthetic apertures. Our paper explores this design space. We compare classical shape from stereo with shape from synthetic aperture focus. We also describe two variants of multi-view stereo based on color medians and entropy that increase robustness to occlusions. We present an experimental comparison of these cost functions on complex light fields, measuring their accuracy against the amount of occlusion.



Figure 1: One image from light field. The inset shows the CD case behind the plants. We compare the performance of different depth measures in reconstructing the depth and color of the CD case.


Figure 2: Histogram showing performance of four depth measures on the CD case.


Adobe Acrobat PDF (181 KB)

Supplementary Web Page

Detailed Results