Computational camera and illumination techniques for recovering "invisible" phenomenon
Date
2016
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
The problem of modeling and reconstructing the "invisibles", e.g., specular or
transparent objects such as 3D
fluid wavefront and gas
flows, has attracted much
attention in recent years. Successful solutions can benefit numerous applications in
oceanology,
fluid mechanism and computer graphics as well as lead to new insights
towards shape reconstruction algorithms. The problem, however, is inherently difficult
for a number of reasons. First such objects do not have their own image. Instead,
they borrow appearance from nearby diffuse objects. Second, determining the light
path within these objects for shape reconstruction is non-trivial since refractions or
reflections non-linearly alter the light paths. Finally, dynamic specular or transparent
objects often exhibit spatially and temporally varying distortions that are hard to
correct. To capture the "invisibles", most previous approaches are based on establishing
point-pixel correspondences. It is well-known that point-pixel correspondences are
under-constrained even for single reflection or refraction. In this dissertation, I propose
to resolve the point-pixel ambiguity by using novel computational imaging devices that
encodes illumination directions.
In particular, I first developed a multi-view based solution for robustly capturing
fast evolving
fluid wavefronts.I constructed a portable camera array system as the main
acquisition device. I elaborately designed the system to allow high-resolution and highspeed
capture and addressed practical issues such as data streaming and storage and
time-divided multiplexing.
Then I exploit using Bokode - a computational optical device that emulates a
pinhole projector - for capturing ray-ray correspondences which can then be used to
directly recover the dynamic
fluid surface normals. I further develop a robust feature
matching algorithm based on the Active Appearance Model (AAM) to robustly establishing
ray-ray correspondences. My solution results in an angularly sampled normal
field and we derive a new angular-domain surface integration scheme to recover the
surface from the normal fields.
I also show another novel computational imaging solution to recover the dynamic
gas
flow by exploiting the light field probe (LF-Probe). A LF-probe resembles a viewdependent
pattern where each pixel on the pattern maps to a unique ray. By observing
the LF-probe through the gas
flow, I acquire a dense set of ray-ray correspondences
and then reconstruct their light paths. To recover the RIF, I use Fermat's Principle
to correlate each light path with the RIF via a Partial Differential Equation (PDE).
I then develop an iterative optimization scheme to solve for all light-path PDEs in
conjunction.
Finally, I extend my directional light coding approach to recover the ambient
occlusion (AO) map of an object. In particular, I adopt a compressive sensing framework
that captures the object under strategically coded lighting directions. I show
that this incident illumination field exhibits some unique properties suitable for AO
recovery. Experiments on synthetic and real scenes show that our approach is both
reliable and accurate with significantly reduced size of input.