Search 250 million research papers

Coding techniques in multiview video coding and joint multiview video model

Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2009

Cited by 0

High Performance Multiview Video Coding

Caoyang Jiang · 2017

Following the standardization of the latest video coding standard High Efficiency Video Coding in 2013, in 2014, multiview extension of HEVC (MV-HEVC) was published and brought significantly better compression performance of around 50% for multiview and 3D videos compared to multiple independent single-view HEVC coding. However, the extremely high computational complexity of MV-HEVC demands significant optimization of the encoder. To tackle this problem, this work investigates the possibilities of using modern parallel computing platforms and tools such as single-instruction-multiple-data (SIMD) instructions, multi-core CPU, massively parallel GPU, and computer cluster to significantly enhance the MVC encoder performance. The aforementioned computing tools have very different computing characteristics and misuse of the tools may result in poor performance improvement and sometimes even reduction. To achieve the best possible encoding performance from modern computing tools, different levels of parallelism inside a typical MVC encoder are identified and analyzed. Novel optimization techniques at various levels of abstraction are proposed, non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) in prediction unit (PU), fractional and bi-directional ME/DE acceleration through SIMD, quantization parameter (QP)-based early termination for coding tree unit (CTU), optimized resource-scheduled wave-front parallel processing for CTU, and workload balanced, cluster-based multiple-view parallel are proposed. The result shows proposed parallel optimization techniques, with insignificant loss to coding efficiency, significantly improves the execution time performance. This , in turn, proves modern parallel computing platforms, with appropriate platform-specific algorithm design, are valuable tools for improving the performance of computationally intensive applications.

Cited by 0

Distributed video coding of multiview sequences

Thomas Maugey · OpenGrey (Institut de l'Information Scientifique et Technique) · 2010

Depuis 2002, le codage vidéo distribué a connu un véritable essor de par ses résultats théoriques séduisants, et ses applications potentielles attractives. En effet, avec ce mode de compression, toute comparaison inter-image est transférée au décodeur, ce qui implique une baisse considérable de la complexité à l'encodeur, et de plus, un encodage indépendant des caméras dans le cas de compression multi-vues. Cette thèse a pour but de proposer de nouvelles solutions dans le domaine du codage vidéo distribué, et particulièrement dans son application aux systèmes multi-caméra. Ces contributions se présentent sous plusieurs aspects : un nouveau modèle débit-distorsion et sa mise en pratique sur trois probléma- tiques, de nouvelles méthodes de construction de l'information adjacente et enfin une étude approfondie du décodeur des trames Wyner-Ziv. Ces nouvelles approches ont toutes pour but d'améliorer les performances débit-distorsion ou de permettre une compréhension plus précise du comportement du codeur. Celles-ci sont exposées en détail dans ce manuscrit avec au préalable une explication complète du contexte dans lequel elles s'inscrivent.

Cited by 0Open access

LOW-COMPLEXITY ASYMMETRIC MULTIVIEW VIDEO CODING

Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2008

Elsevier’s Scopus, the largest abstract and citation database of peer-reviewed literature. Search and access research from the science, technology, medicine, social sciences and arts and humanities fields.

Cited by 0

Joint multiview video plus depth coding

Jun Zhang, Miska M. Hannuksela, Houqiang Li · 2010

Multiview video plus depth (MVD), where per-pixel depth map sequences are associated with multiview texture video, is a promising approach for three-dimensional (3D) video solutions requiring view synthesis. The Multiview Video Coding (MVC) standard has been typically used to compress the MVD representation, resulting into the multiview texture video and the multiview depth video being in two separate MVC bitstreams. Such a coding scheme does not, however, utilize the similarities of the motion information of the multiview texture video and the multiview depth video. The joint multiview video plus depth coding (JMVDC) scheme presented in this paper uses inter-view prediction similarly to MVC as well as the inter-layer motion prediction tool of the Scalable Video Coding (SVC) standard to exploit the correlation of the motion in the texture and depth video sequences. The simulation results show that the JMVDC method achieves about 10 to 20% saving in depth bitrate compared with conventional MVC-based coding of MVD representations.

Cited by 32

Scalable multiview video coding using wavelet

Feng Wu, Yan Lu, Jianfei Cai et al. · 2005

Abstract—Most existing multiview video coding (MVC) techniques are based on the traditional hybrid DCT-based video coding schemes, e.g., MPEG-2, MPEG-4 and H.264. They neither fully exploit the redundancy among different views nor provide an easy way of implementation for scalabilities. In this paper, we propose an MVC scheme based on wavelet, which can provide temporal, spatial, SNR as well as view scalabilities. To the best of our knowledge, wavelets have not been used for MVC in the literature before. In particular, we consider a multiview video as a 2D matrix with 1D along the temporal axis, 1D along the view axis, and each element in the 2D matrix represents a video frame. We first apply 1D wavelet decomposition to the 2D matrix along the temporal axis with motion compensation, and then along the view axis with disparity compensation. After that, 2D spatial wavelet decomposition is applied to each element in the matrix. Finally, all the subbands are encoded by 3D-ESCOT with rate-distortion optimization. Compared with traditional 3D wavelet coding, our proposed wavelet-based MVC scheme applies one more dimensional wavelet transform along the view direction to exploit the redundancy between adjacent views. Some preliminary experiments are carried out using standard multiview video sequences and the efficiency of the proposed system is confirmed. I.

Cited by 0

Wyner–Ziv-Based Multiview Video Coding

Xun Guo, Yan Lu, Feng Wu et al. · IEEE Transactions on Circuits and Systems for Video Technology · 2008

Utilizing video correlations among views would definitely improve multiview video compression in terms of coding efficiency, which usually requests an expensive system to collect videos from different cameras and jointly compress them. Thanks to recent developments on distributed video coding, this paper proposes a new multiview video coding scheme based on Wyner-Ziv (WZ) coding technique, in which the complicated temporal and interview correlation exploration process is shifted from the encoder side to the decoder side so that broadband raw data traffic and high intensive computation for jointly encoding can be avoided. At the encoder side, a wavelet-based WZ scheme is proposed to compress video of every camera. Furthermore, in order to better utilize correlation in wavelet domain, all coefficients are organized as that done in SPIHT bit plane by bit plane. At the decoder side, a more flexible prediction technique that can jointly utilize temporal and view correlations is proposed to generate side information. Finally, experimental results show the proposed scheme significantly outperforms the conventional intra-frame coding for better random access and is even close to the inter-frame coding for better efficiency. Furthermore, compressed data is much robust when it is transmitted over an error-prone channel.

Cited by 66

Multiview Video Coding for Virtual Reality

Kashyap Kammachi Sreedhar · Tampere University Institutional Repository (Tampere University) · 2016

Virtual reality (VR) is one of the emerging technologies in recent years. It brings a sense of real world experience in simulated environments, hence, it is being used in many applications for example in live sporting events, music recordings and in many other interactive multimedia applications. VR makes use of multimedia content, and videos are a major part of it. VR videos are captured from multiple directions to cover the entire 360 field-of-view. It usually employs, multiple cameras of wide field-of-view such as fisheye lenses and the camera arrangement can also vary from linear to spherical set-ups. Videos in VR system are also subjected to constraints such as, variations in network bandwidth, heterogeneous mobile devices with limited decoding capacity, adaptivity for view switching in the display.
\n
\nThe uncompressed videos from multiview cameras are redundant and impractical for storage and transmission. The existing video coding standards compresses the multiview videos effi ciently. However, VR systems place certain limitations on the video and camera arrangements, such as, it assumes rectilinear properties for video, translational motion model for prediction and the camera set-up to be linearly arranged.
\n
\nThe aim of the thesis is to propose coding schemes which are compliant to the current video coding standards of H.264/AVC and its successor H.265/HEVC, the current state-of-the-art and multiview/scalable extensions. This thesis presents methods that compress the multiview videos which are captured from eight cameras that are arranged spherically, pointing radially outwards. The cameras produce circular fi sheye videos of 195 degree field-of-view. The final goal is to present methods, which optimize the bitrate in both storage and transmission of videos for the VR system.
\nThe presented methods can be categorized into two groups: optimizing storage bitrate and optimizing streaming bitrate of multiview videos. In the storage bitrate category, six methods were experimented. The presented methods competed against simulcast coding of individual views. The coding schemes were experimented with two data sets of 8 views each. The method of scalable coding with inter-layer prediction in all frames outperformed simulcast coding with approximately 7.9%. In the case of optimizing streaming birates, five methods were experimented. The method of scalable plus multiview skip-coding outperformed the simulcast method of coding by 36% on average.
\n
\nFuture work will focus on pre-processing the fi sheye videos to rectilinear videos, in-order to fit them to the current translational model of the video coding standards. Moreover, the methods will be tested in comprehensive applications and system requirements.

Cited by 1Open access

Multiview video coding for 3D telepresence

Wenxian Yang · 2006

Telepresence refers to a human/machine system in which the human operator feels physically present at a remote site. Telepresence systems have high potential in various applications of consumer entertainment.

Cited by 0Open access

DIAGONAL INTERVIEW PREDICTION FOR MULTIVIEW VIDEO CODING

Ping Yang, Yun He · 2013

Multi-view video receives many attentions in these years, because it can support a wide range of applications, such as 3D video communication and free view point video. To improve the coding efficiency, multi-view video coding not only employs temporal predictions but also predictions between different views. This paper investigates the parallel computing, low delay and scalability problems induced from the temporal-spatial predictions, proposes a diagonal interview prediction (DIP) structure. It is shown in the test results that the DIP can exploit the correlations between different views largely, it can support parallel computing, low delay and scalable decoding features, and it obtains a high coding gain especially for the large motion cases. Index Terms — DIP, multi-view video, prediction structure, low delay

Cited by 0

Geometric prediction structure for multiview video coding

Seok Lee, Ho-Cheon Wey, Du‐Sik Park · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2010

One of the critical issues to successful service of 3D video is how to compress huge amount of multi-view video data efficiently. In this paper, we described about geometric prediction structure for multi-view video coding. By exploiting the geometric relations between each camera pose, we can make prediction pair which maximizes the spatial correlation of each view. To analyze the relationship of each camera pose, we defined the mathematical view center and view distance in 3D space. We calculated virtual center pose by getting mean rotation matrix and mean translation vector. We proposed an algorithm for establishing the geometric prediction structure based on view center and view distance. Using this prediction structure, inter-view prediction is performed to camera pair of maximum spatial correlation. In our prediction structure, we also considered the scalability in coding and transmitting the multi-view videos. Experiments are done using JMVC (Joint Multiview Video Coding) software on MPEG-FTV test sequences. Overall performance of proposed prediction structure is measured in the PSNR and subjective image quality measure such as PSPNR.

Cited by 0

4-D Wavelet-Based Multiview Video Coding

Wenxian Yang, Yan Lü, Feng Wu et al. · IEEE Transactions on Circuits and Systems for Video Technology · 2006

The conventional multiview video coding (MVC) schemes, utilizing both neighboring temporal frames and view frames as possible references, have only shown a slight gain over those using temporal frames alone in terms of coding efficiency. The reason for this is that the neighboring temporal frames exhibit stronger correlation with the current frame and the view frames often fail to be selected as references. This paper proposes an elegant MVC framework using high dimensional wavelet, which rightly matches the inherent high dimension property of multiview video. It also makes a better usage of both temporal and view correlations thanks to the hierarchical decomposition. Besides the proposed framework, this paper also investigates MVC coding from the following aspects. First, a disparity-compensated view filter (DCVF) with pixel alignment is proposed, which can accommodate both global and local view disparities among view frames. The proposed DCVF and the existing motion-compensated temporal filter (MCTF) unify the view and temporal decompositions as a generic lifting transform. Second, an adaptive decomposition structure based on the analysis of the temporal and view correlations is proposed. A Lagrangian cost function is derived to determine the optimum decomposition structure. Third, the major components of the proposed MVC coding are figured out, including macroblock type design, subband coefficient coding, and rate allocation. Extensive experiments are carried out on the MPEG 3DAV test sequences and the superior performance of the proposed MVC coding is demonstrated. In addition, the proposed MVC framework can easily support temporal, spatial, SNR, as well as view scalabilities

Cited by 47

Scalable multiview video coding for immersive video streaming systems

Hoda Roodaki, Shervin Shirmohammadi · 2016

Immersive video places the user inside the video scene, allowing the user to control the direction of the view. To achieve this, the view of every direction must be recorded using either a panoramic camera or multiple cameras placed at different positions with different angels. The size of the captured video can be quite large due to multiple video streams, one from each camera. Even with compression standards such as Multiview Video Coding (MVC), the transmission of the whole MVC video is still bandwidth-costly, especially for heterogeneous users whose bandwidths vary. In this paper, we present a new approach for immersive video streaming by using Scalable Multiview Video Coding (SMVC) to create multiple layers of the immersive video, supporting heterogeneous receivers more efficiently. Our method limits the number of views in its base layer, while it uses view scalability and free view-point scalability in the additional layers to synthesize more views at the receiver and provide high quality free view-point viewing to the user. Performance evaluations demonstrate that our method: 1-synthesizes missing views more accurately, as evident subjectively, and 2-achieves an average and maximum gain of 0.75 and 1.4 in Bjontegaard BD-Bitrate scale, respectively, compared to existing work which simply group adjacent views in the same layer.

Cited by 2

Fusion schemes for multiview distributed video coding

Maugey T., Miled W., Cagnazzo M. et al. · Research Padua Archive (University of Padua) · 2009

Distributed video coding performances strongly depend on the side information quality, built at the decoder. In multi-view schemes, correlations in both time and view directions are exploited, obtaining in general two estimations that need to be merged. This step, called fusion, greatly affects the performance of the coding scheme; however, the existing methods do not achieve acceptable performances in all cases, especially when one of the estimations is not of good quality, since in this case they are not able to discard it. This paper provides a detailed review of existing fusion methods between temporal and inter-view side information, and proposes new promising techniques. Experimental results show that these methods have good performances in a variety of configurations. © EURASIP, 2009.

Cited by 0

Fast prediction algorithm for multiview video coding

Abdelrahman Abdelazim, Stephen James Mein, M.R. Varley et al. · Optical Engineering · 2013

he H.264/multiview video coding (MVC) standard has been developed to enable efficient coding for three-dimensional and multiple viewpoint video sequences. The inter-view statistical dependencies are utilized and an inter-view prediction is employed to provide more efficient coding; however, this increases the overall encoding complexity. Motion homogeneity is exploited here to selectively enable inter-view prediction, and to reduce complexity in the motion estimation (ME) and the mode selection processes. This has been accomplished by defining situations that relate macro-blocks’ motion characteristics to the mode selection and the inter-view prediction processes. When comparing the proposed algorithm to the H.264/MVC reference software and other recent work, the experimental results demonstrate a significant reduction in ME time while maintaining similar rate-distortion performance.

Cited by 4

Optimized Coding Strategies for Interactive Multiview Video

De Abreu Goes · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · 2015

The natural next step in improving the realistic experience in multimedia services is interactive multiview video (IMV). IMV promises to enable the users to freely navigate through a scene by selecting their preferred viewpoints from any view position for which the corresponding view is generated. A smooth navigation could be achieved with camera views and views synthesized at the decoder. However, the large amount of data required for such navigation experience still represents a challenge for the current systems, which implies the need for new efficient coding strategies that permit to save on storage and transmission resources, while preserving interactivity in the navigation. In this thesis, we focus on the optimization of coding strategies for IMV systems. First, we address the issues related to the coding techniques for IMV in a multiview video plus depth (MVD) scenario, where texture and depth maps are available for view synthesis at the decoder. We propose a low complexity algorithm for the selection of the interview prediction structures (PSs) and associated texture and depth quantization parameters (QPs) for IMV under transmission and storage constraints. Simulation results show that our novel low complexity algorithm has near-optimal compression efficiency while preserving interactivity properties at the decoder. Then, considering the limited and heterogeneous capabilities of current networks and decoding devices, we propose a novel adaptive solution for IMV based on a layered multiview representation where camera views are organized into layered subsets to offer different levels of navigation quality depending on the different client constraints. We propose an optimal and a reduced computational complexity greedy algorithms that jointly select the different view subsets and their encoding rates. Simulation results show the good performance of our novel algorithms compared to a baseline algorithm, proving that an effective IMV adaptive solution should consider the scene content, the client capabilities and their preferences, in building adaptive systems for multiview navigation. Finally, we build on the solution proposed in our second problem and present a general solution to rate allocation problems in multiview video. In particular, we propose a new algorithm to find the optimal Lagrange multiplier in a Lagrangian-based rate allocation problem. We show the performance of our proposed algorithm in both multiview and monoview video scenarios and show that the proposed method is able to compete with complex state-of-the-art rate control techniques. In summary, this thesis addresses important issues for coding multiview video in the design of efficient IMV systems under resource constraints. Our algorithm to select the optimal PS and QPs in a MVD scenario can improve the quality of the rendered views and it can indeed provide new insights for a deeper understanding of specific IMV coding requirements. We show that our algorithm for a layered representation of multiview video provides an effective adaptive streaming solution for IMV systems with users with limited and heterogeneous capabilities. Finally, our proposed Lagrangian-based rate allocation algorithm with an optimized selection of the Lagrange multiplier represents a general contribution that can be used in multiple video scenarios.

Cited by 2Open access

Low-complexity scalable and multiview video coding

Sebastiaan Van Leuven · Ghent University Academic Bibliography (Ghent University) · 2013

collega's en voormalig collega's bedankt voor de samenwerking, de mogelijkheden die gecrerd zijn en de talloze discussies die ervoor gezorgd hebben dat er steeds nieuwe inzichten kwamen.Hierbij had

Cited by 0Open access

Dense disparity estimation in multiview video coding

Ismaël Daribo, Mounir Kaaniche, Wided Miled et al. · 2009

Multiview video coding is an emerging application where, in addition to classical temporal prediction, an efficient disparity prediction should be performed in order to achieve the best compression performance. A popular coder is the multiview video coding (MVC) extension of H.264/AVC, which uses a block-based disparity estimation (just like temporal prediction in H.264/AVC). In this paper, we propose to improve the MVC extension by using a dense estimation method that generates a smooth disparity map with ideally infinite precision. The obtained disparity is then segmented and efficiently encoded by using a rate-distortion optimization technique. Experimental results show that significant gains can be obtained compared to the block-based disparity estimation technique used in the MVC extension.

Cited by 9

Color Correction Preprocessing for Multiview Video Coding

Colin Doutre, Panos Nasiopoulos · IEEE Transactions on Circuits and Systems for Video Technology · 2009

In multiview video, a number of cameras capture the same scene from different viewpoints. There can be significant variations in the color of views captured with different cameras, which negatively affects performance when the videos are compressed with inter-view prediction. In this letter, a method is proposed for correcting the color of multiview video sets as a preprocessing step to compression. Unlike previous work, where one of the captured views is used as the color reference, we correct all views to match the average color of the set of views. Block-based disparity estimation is used to find matching points between all views in the video set, and the average color is calculated for these matching points. A least-squares regression is performed for each view to find a function that will make the view most closely match the average color. Experimental results show that when multiview video is compressed with joint multiview video model, the proposed method increases compression efficiency by up to 1.0 dB in luma peak signal-to-noise ratio (PSNR) compared to compressing the original uncorrected video.

Cited by 57

Multiview-Video-Plus-Depth Coding Based on the Advanced Video Coding Standard

Miska M. Hannuksela, Moncef Gabbouj, Dmytro Rusanovskyy et al. · IEEE Transactions on Image Processing · 2013

This paper presents a multiview-video-plus-depth coding scheme, which is compatible with the advanced video coding (H.264/AVC) standard and its multiview video coding (MVC) extension. This scheme introduces several encoding and in-loop coding tools for depth and texture video coding, such as depth-based texture motion vector prediction, depth-range-based weighted prediction, joint inter-view depth filtering, and gradual view refresh. The presented coding scheme is submitted to the 3D video coding (3DV) call for proposals (CfP) of the Moving Picture Experts Group standardization committee. When measured with commonly used objective metrics against the MVC anchor, the proposed scheme provides an average bitrate reduction of 26% and 35% for the 3DV CfP test scenarios with two and three views, respectively. The observed bitrate reduction is similar according to an analysis of the results obtained for the subjective tests on the 3DV CfP submissions.

Cited by 106