Texture and mid-level features of objects and scenes in images

Abstract

Features of images have been widely explored in the literature to better represent and utilize images. The development of deep learning has brought new challenges to image feature representations that they shall be easy to build and captures as much as information with minimum space. In this thesis, multiple aspects of image features have been studied to give a better utilization of textures and mid-level representations of objects and scenes in images.

First we investigate texture synthesis of art patterns and line drawings. Inspired by exemplar-based texture synthesis methods, we bring up a global energy function which considers both the appearance and curvilinear feature similarities between the input exemplar and the output image. An EM-like scheme is performed to optimize that energy function, which can be easily to extend to handle multi-layer textures. The remaining part of this thesis is about creating mid-level features that can be used to classify objects and scenes in images more effectively. We propose a multi-layer deep learning model, SCaLE, to achieve intermediate image representation that can be directly used by nearest neighbor classifier. We then propose to use meta-objects and build a novel scene classification pipeline. Both models work well and give state-of-the-art performance on benchmark datasets.