Computational image reconstruction via data-driven and model-based algorithms

Abstract

Computational imaging (CI) refers to the joint design of physical imaging systems and computational methods for post-image processing based on prior imperfect recording of scenes. This integrated paradigm aims at improving the imaging performance or extracting implicit information from the captured raw measurements via computation. By introducing an extra computational step to the imaging, the burden brought by unwieldy and expensive hardwares, physical limits on optics, or stringent requirements of the experimental process, can be alleviated and transferred to computation, yielding various new designs of instruments with reduced restrictions on size, weight, cost, fabrications or acquisition environment.

Image reconstruction has always been an essential task in CI studies as the raw intensity images are usually not ready for direct perception or non-interpretable with no explicit fidelity to the imaged scene. In general, computational image reconstruction includes a series of processing problems like denoising, deconvolution, aberrations correction and so on. The traditional model-based algorithms, relying on the explicit knowledge of the image formation process, often suffer from various artifacts bought by hand-picked parameters, experimental errors, imprecise approximation of physical models, etc. This dissertation is in pursuit of handling computational image reconstruction problems using learning-based algorithms and also hybrid architectures incorporating both model-based and data-driven methods.

To reconstruct clean images from measurements captured via coherent imaging, it is especially critical to suppress the speckle noise, caused by the scattering of coherent light sources after irradiating rough surfaces. By combining nonlocal self-similarity (NSS) filters with machine learning, which makes use of convolutional neural network (CNN) denoisers, a modular framework is developed to tackle the reconstruction of images corrupted with speckles. Besides noticeable improvements in experimental results, the plug-and-play design enables the direct usage of different models without re-training, making the proposed framework more adaptable.

Model mismatch is also a common problem that leads to inferior image reconstruction results. In mask-based lensless imaging, the attributing factors to model mismatch error are quite complicated and diverse, varying from occlusion to sensor imperfections, from large impinging angles to specular objects. To overcome this, a novel physics-informed deep learning architecture with focus on correcting such errors is developed. The proposed hybrid reconstruction network consists of both unrolled model-based optimization to apply system physics and deep learning layers for model correction. Experimental results have demonstrated the effectiveness and robustness of the proposed architecture.

As an emerging technique in deep learning, the capsule network is designed to overcome information loss in the pooling operation and internal data representation of CNNs. It has shown promising results in several applications, such as digit recognition and image segmentation. Thus, we investigate for the first time the use of a capsule network in digital holographic reconstruction. The proposed residual encoder-decoder capsule network, which we call RedCap, uses a novel windowed spatial dynamic routing algorithm and residual capsule block. Compared with the CNN-based network, RedCap exhibits much better experimental results while having a dramatic 75% reduction in the number of parameters, indicating that RedCap is more efficient in the way it processes data.