Novel techniques for depth map compression

Abstract

Virtual reality will be an important application of modern communication systems. 3D video technologies are important options for enhancing user experience by providing changeable viewpoints under reasonable complexity. 2D or multiview plus depth video formats are attractive because of their simplicities in providing such experience at relatively low data bandwidth. The first standard for coding such videos is the ITU-T H.265 3D-HEVC standard, where coding tools were developed to explore the constant and piecewise nature of depth maps. It also makes use of the coded texture for joint texture-depth coding. This thesis aims to develop new techniques and a coder/decoder (codec) for depth map sequences compression without referencing the coded texture. An advantage is that depth maps can be separately encoded/decoded and used with any video coding standards. The proposed codec explores geometrical models and adaptive quantisation techniques for intra-frame coding and uses motion compensation/interpolation techniques for inter-frame coding. In particular, we proposed three local models for intra-frame compression. The first set of models is based on the observation that many depth maps are generated by computer graphics and hence are locally smooth. Thus, they can be approximated by spline and polynomial functions. A bilinear interpolation-based method is proposed where three previously coded pixels and an additional pixel are used to define the bilinear surface inside a block. Moreover, this surface can be efficiently reconstructed without multiplication. A 'global planes' method is also introduced to model possible 3D planes in the scene, which can be applicable to multiple blocks. The second model is an edge-based model, which aims to represent sharp discontinuities in depth map found at boundaries between foreground and background. It extends the Wedgelet model, which represents sharp changes with a straight line inside a block, by introducing turning points to form a more flexible polygonal boundary. The positions of turning points are coded, through which simple discontinuities can be efficiently restored. The third model is a layer-based model, which explores the layered structures of depth maps estimated via computer vision techniques. It quantises the depth values inside a block into several layers with constant values. The layers are then compressed using context-adaptive binary arithmetic coding. Since this method does not assume any geometrical model, it can be used as a general method for encoding complicated scenes. The proposed layer-based method becomes prevalent in highly irregular depth maps as an effective way to reconstruct complicated structures. Based on these and conventional intra-frame and inter-frame coding methods, a new codec for depth map sequences was developed. Coding results on standard testing sequences demonstrated the improved performance of the proposed at low to medium bit rates over conventional methods, especially for computer generated and high quality depth videos. Moreover, a real-time decoder and an accelerated encoder of the proposed codec were successfully implemented in a personal computer with Intel Core i7-4770 3.40 GHz CPU and NVIDIA GeForce GTX 1080 Ti GPU.