Nonparametric Bayesian methods for visual data association

Abstract

Visual data association, widely used in object modeling among multi-dimensional visual data, proves both useful and challenging. Manual preprocessing of object tracking is difficult and costly, involving procedures such as labeling the object position and setting constraint for the tracking trajectory. Massive computational cost and inconsistency in the number of objects identified also cause troubles in actual use. In this dissertation, emphasis is placed on three aspects, namely object vari- ations among regions in two-dimensional (2D) images, object trajectories among frames in three-dimensional (3D) videos and object relationships among view- points in the four-dimensional (4D) light field, which are addressed with spatial, temporal, and light field nonparametric Bayesian methods respectively. First of all, the Doubly Stochastic Dirichlet Process (DSDP) is proposed in the global topic measurement space modeling, which poses weaker assumption com- pared to the discrete Markov assumption, thus resulting in a lower computational cost than other nonparametric Bayesian models. A mixture model of the spatial DSDP is also presented, thinned from Dirichlet Process Mixture (DPM) with- out considerable auxiliary covariates, where the marked function prior makes the number of land-cover classes consistent while the stochastic process prior modelsthe 2D hyperspectral image (HSI) land cover variation globally. The consistency of the number of land covers is maintained in large-scale geographical areas of the HSI. Experiments show that the model is robust and consistent for the HSI identification problem with weak or even no supervision. Second, research is conducted over unsupervised tracking algorithm for hu- man and car trajectories detection in 3D video clips through the mixture model of temporal Doubly Stochastic Dirichlet Process (TDSDP). The TDSDP cap- tures the varying flow of crowds and vehicles in the temporal domain without the Markov assumption, making it particularly suitable for long-term tracking. Besides, TDSDP prior can estimate the number of trajectories automatically. Experimental results using synthetic and real-world data show that the proposed TDSDP mixture is superior to the DPM and Dependent Dirichlet Process (DDP) concerning topic variation modeling. PETS2001 dataset experiments show that TDSDP has more robust object tracking capability over DDP based on General- ized Polya Urn.

Third, a sparse hierarchical nonparametric Bayesian model is used to repre- sent the data captured by 4D light field cameras, in which the light field can be regarded as a set of sub-aperture views. To capture the visual variations of the object in different viewpoints, a concept called “depth flow features” is proposed. These views can then be modeled statistically with a sparse representation in a fully unsupervised manner. While local dictionaries are learned from the corre- sponding sub-aperture view, all the views with different perspectives share one global dictionary. To prove the validity of the proposed model, it is applied to denoise the light field data. It is demonstrated that this method outperforms several state-of-the-art approaches in this application. (460 words)