Neuromorphic event sensing for laser speckle imaging and autofocusing microscopy

Abstract

Imaging systems primarily rely on photodetectors to monitor and capture light. Conventional image sensors such as charge-coupled device (CCD) and complementary metal oxide semiconductor (CMOS) are almost always used in various imaging systems to record the absolute intensity of every pixel. However, such sensing scheme is not good enough to carry dynamic information, especially in some imaging systems that are designed for moving scenes. Inspired by biology retinas, neuromorphic event sensor is emerging as a promising sensing paradigm where each pixel works independently to detect brightness changes at high temporal resolution and low latency. The output is a stream of asynchronous event data that carriesonly binary information of the brightness changes, together with the pixel location and timestamp. Such unique characteristics drastically reduce the amount of generated data and improve the efficiency in both data acquisition and subsequent computation. The purpose of this dissertation is to harness the neuromorphic event sensing paradigm to cope with the challenges in laser speckle imaging (LSI) and autofocusing microscopy. First, we tackle the problem of dynamic laser speckle analysis (DLSA) which is a non-contact method to estimate the dynamic levels of the inspected objects. The sample surface is illuminated using a coherent light source and the results can be obtained by analyzing the series of reflected speckle patterns. As the first reported work in DLSA with neuromorphic event sensing, we present the proof-of-principle results and demonstrate the superiority of our method compared with conventional frame-based image sensors under different dynamic levels. Two evaluation metrics are proposed in this work to efficiently analyze the event form of speckle patterns. Second, we propose a neuromorphic LSI method to estimate micro motion. Traditional LSI techniques rely on identifying changes from the timecorrelated intensity speckle patterns, where a lot of redundant data of the static speckles without motion information will also be recorded and the motion cues are inevitably lost during the “blind” time interval between successive frames. In our proposed method, the reflected high frequency laser speckle patterns are captured using a neuromorphic event sensor on the order of microseconds. We present two data processing strategies that are based on speckle correlation and block matching to estimate micro motion from the event-based laser speckles. Experimental results demonstrate the validity and robustness of our method in a wide range of motion speeds with less than 2% error rate. Third, we focus on the autofocusing problem in microscopy. Rapid autofocusing is essential for many microscopic imaging applications. Existing methods either require complicated hardware implementations or slow z-stack image acquisition. To cope with this problem, we develop a new approach to achieve fast autofocusing by detecting the brightness variation in the axial diffraction using the neuromorphic event sensing. A simple yet efficient autofocusing system is tailored for fast acquisition and processing of the non-redundant event data, allowing for autofocusing in only tens of milliseconds, which is thousands of times faster than current technologies. Experimental results show a substantial performance improvement and capability for biopsy specimen inspections.