An efficient multiview video coding system with application in AVS

Abstract

Multiview video coding (MVC) systems have attracted much attention because of their applications in free view point television (FTV), three-dimensional television (3DTV), surveillance systems, etc. Compared with the traditional 2D media, multiview videos usually involve large data size. It is essential to develop high performance MVC systems with low computational complexity. This thesis aims to develop an efficient multiview video codec based on the Audio Video Coding Standard of China (AVS) and investigate its application to image-based rendering (IBR).

Firstly, a new fast motion estimation (ME) algorithm based on the simplified unsymmetrical hexagonal method (UMHexagonS) used in the AVS standard is developed. Different from the conventional method, the proposed approach uses an extended searching area and a square-based search to reduce computation complexity considerably. To obtain real-time implementation, we proposed a macroblock-based parallel structure which utilizes multi-core processing. Such approach can improve the encoding frame rate by approximately 4 times compared to traditional implementations. The proposed method has been tested on an Intel Core i7 3960x six cores 3.3GHz CPU. The real-time encoding speed reaches 25 frames per second (fps) for a 1280×720 high-definition video.

Secondly, a multiview video coding algorithm based on AVS which encode three views as a group is developed. By taking advantages of the reduced inter-view dependency among adjacent views, the proposed algorithm can simplify the selective transmission and the decoding of multiview videos. Moreover, a new disparity estimation (DE) algorithm based on an unsymmetrical search window is proposed to further reduce the computational complexity. The developed encoder can achieve real-time encoding for 640×480 3-view videos at more than 26fps on a computer equipped with an Intel Core i7 3960x six cores 3.3GHz CPU.

Finally, an image-based rendering and object-based compression system based on the Kinect depth camera and algorithm are developed for indoor applications. The proposed system uses the depth information obtained by the Kinect depth camera together with the texture information to perform IBR. Experimental results illustrate that good performance and usefulness can be achieved by the proposed approach.