LED-based snapshot compressive 4D spectral temporal imaging
Date
2022
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
High frame-rate video recording poses challenging hardware requirements such as faster sensor readout speed, larger memory buffers, and broader bandwidths. Snapshot compressive imaging solves these problems via sampling the high-dimensional image cubes containing high frame-rate multi-spectral images, with designed coding, onto a single 2D snapshot, then recovering the image signal after the measurement. In this research work, a new LED-based compressive spectral temporal imaging (LeSTI) system is designed with optimal encoding. Temporal and spectral resolution is further enhanced by deep neural network algorithms. ☐ A multi-spectral Light-Emitting-Diode (LED) array is used for target illumination and spectral modulation while a Digital Micro-mirror Device (DMD) encodes the spatial and temporal frames. Several encoded video frames are captured in a snapshot of an integrating focal plane array (FPA).A high-frame-rate spectral video is reconstructed from the sequence of compressed measurements captured by the grayscale low-frame-rate camera. The imaging system is optimized through the design of the DMD patterns based on the forward model. Laboratory implementation is conducted to validate the performance of the proposed imaging system. We experimentally demonstrate the video acquisition with eight spectral bands and six temporal frames per FPA snapshot, and thus a 256x256x8x6 4D cube is reconstructed from a single 2D measurement. ☐ Furthermore, we find out multi-spectral imaging usually requires scanning in spatial or spectral dimensions and thus the temporal resolution is often compromised. In this dissertation, we propose a deep learning based algorithm for LeSTI system to overcome the scanning requirement in spectra, coined as LeSTI++. The proposed algorithm allows us to capture high speed moving targets without compromising temporal resolution. The reconstruction results show high resolution in spectral and temporal with efficiency. We compare the proposed algorithm with other state-of-the-art in simulation, and verify the algorithm with experimental data. With the newly proposed imaging model and the reconstruction algorithm a 256x256x25x24$ 4D cube is reconstructed from a single 2D measurement.
Description
Keywords
Compressive imaging, LED, Spectral-temporal imaging