Data-adaptive graph-regularized matrix factorizations

Abstract

Matrix factorizations have been utilized in many fields, such as recommendation system, genetic research, and image processing. They are used for exploring hidden pattern from observed data, recovering missing data and predicting unknown scenarios. Due to the rapidly growing amount of data and increasing demands on data analysis capability, side information is widely employed on top of the basic matrix factorization models. In particular, graph information, which provides data correlations or pairwise similarities, is one commonly-used side information. Due to the additional perspective provided by the graph information, graph-regularized matrix factorization models are proved to outperform vanilla matrix factorization models in various applications, including matrix completion and multi-label learning.

Invoking graph information in matrix factorizations requires the model to balance information from data observation, graph regularizations and model assumptions. However, the data-adaptability of regularization is often overlooked in existing graph-regularized matrix factorization works. More specifically, matrix factorization gives rise to the concept of theme as it groups data into multiple components in an unsupervised learning manner. Existing graph-regularized works ignore the theme-wise data partition and apply the same regularization parameters to all themes. While generalizing to theme-wise graph regularization may seem straightforward in the first sight, it leads to a dramatically increased number of parameters to be tuned in traditional optimization-based methods. Besides the theme-wise adaptability, data-adaptive regularizations become necessary if the graph information varies in different parts of the data. In this case, the graph regularizations should be embedded locally, whose mechanism also needs further investigation.

To allow theme-wise adaptive regularization and meanwhile overcome the computational costly regularization parameter tunning, this thesis investigates the graph-regularized matrix factorization problem from the probabilistic model perspective. Benefit from the newly designed prior distribution and Bayesian inference techniques, the graph regularization can be adaptively embedded theme-wisely, and at the same time, the regularization parameters are automatically learned. Above that, model assumptions such as low-rankness can be simultaneously incorporated into the proposed model to tackle the missing data challenge. A nontrivial conditional conjugacy is exploited such that an efficient probabilistic graph-regularized matrix factorization algorithm can be derived under variational inference framework. Extensive numerical results on various matrix completion and multi-label learning applications show the superior performance of the proposed tuning-free method compared to existing state-of-the-art models.

In addition to graph regularized factorization of a single matrix, this thesis also investigates factorization of multiple but coupled matrices in the context of functional magnetic resonance imaging. In this case, graph information represents the structural brain connections that can be leveraged to learn the functional interactions between different regions of the brain. This structurally-informed factorization has long been a challenge as the structural brain connections have inherent sparsity and vary for each subject. To overcome this challenge, a novel sparse non-negative matrix factorization model with local graph regularizations is proposed. Results show that the estimated latent dynamic functional brain networks are closer to a priori knowledge of brain organization and acquire better interpretability due to the elimination of redundant bases for the latent functional brain networks.