Realistic representations using extremely high quality images are becoming increasingly popular. For example, digital cinemas can now display moving pictures composed of high-resolution digital images. Although these applications focus on increasing the spatial resolution only, higher frame-rates are being considered to achieve more realistic representations. Since increasing the frame-rate increases the total amount of information, efficient coding methods are required. However, its statistical properties are not clarified. This paper establishes for high frame-rate video a mathematical model of the relationship between frame-rate and bit-rate. A coding experiment confirms the validity of the mathematical model.

Recently, image compression systems based on convolutional neural networks that use flexible nonlinear analysis and synthesis transformations have been developed to improve the restoration accuracy of decoded images. Although these methods that use objective metric such as peak signal-to-noise ratio and multi-scale structural similarity for optimization attain high objective results, such metric may not reflect human visual characteristics and thus degrade subjective image quality. A method using a framework called a generative adversarial network (GAN) has been reported as one of the methods aiming to improve the subjective image quality. It optimizes the distribution of restored images to be close to that of natural images; thus it suppresses visual artifacts such as blurring, ringing, and blocking. However, since methods of this type are optimized to focus on whether the restored image is subjectively natural or not, components that are not correlated with the original image are mixed into the restored image during the decoding process. Thus, even though the appearance looks natural, subjective similarity may be degraded. In this paper, we investigated why the conventional GAN-based compression techniques degrade subjective similarity, then tackled this problem by rethinking how to handle image generation in the GAN framework between image sources with different probability distributions. The paper describes a method to maximize mutual information between the coding features and the restored images. Experimental results show that the proposed mutual information amount is clearly correlated with subjective similarity and the method makes it possible to develop image compression systems with high subjective similarity.

In current video encoding systems, the acquisition process is independent from the video encoding process. In order to compensate for the independence, pre-filters prior to the encoder are used. However, conventional pre-filters are designed under constraints on the temporal resolution, so they are not optimized enough in terms of coding efficiency. By relaxing the restriction on the temporal resolution of current video encoding systems, there is a good possibility to generate a video signal suitable for the video encoding process. This paper proposes a video generation method with an adaptive temporal filter that utilizes a temporally over-sampled signal. The filter is designed based on dynamic-programming. Experimental results show that the proposed method can reduce encoding rate on average by 3.01 [%] compared to the constant mean filter.

Designing an optimum quantizer can be treated as the optimization problem of finding the quantization indices that minimize the quantization error. One solution to the optimization problem, DP quantization, is based on dynamic programming. Some applications, such as bit-depth scalable codec and tone mapping, require the construction of multiple quantizers with different quantization levels, for example, from 12bit/channel to 10bit/channel and 8bit/channel. Unfortunately, the above mentioned DP quantization optimizes the quantizer for just one quantization level. That is, it is unable to simultaneously optimize multiple quantizers. Therefore, when DP quantization is used to design multiple quantizers, there are many redundant computations in the optimization process. This paper proposes an extended DP quantization with a complexity reduction algorithm for the optimal design of multiple quantizers. Experiments show that the proposed algorithm reduces complexity by 20.8%, on average, compared to conventional DP quantization.

It is essential to improve intra prediction performance to raise the efficiency of video coding. In video coding standards such as H.265/HEVC, intra prediction is seen as an extension of directional prediction schemes, examples include refinement of directions, planar extension, filtering reference sampling, and so on. From the view point of reducing prediction error, some improvements on intra prediction for standardized schemes have been suggested. However, on the assumption that the correlation between neighboring pixels are static, these conventional methods use pre-defined predictors regardless of the image being encoded. Therefore, these conventional methods cannot reduce prediction error if the images break the assumption made in prediction design. On the other hand, adaptive predictors that change the image being encoded may offer poor coding efficiency due to the overhead of the additional information needed for adaptivity. This paper proposes an adaptive intra prediction scheme that resolves the trade-off between prediction error and adaptivity overhead. The proposed scheme is formulated as a constrained optimization problem that minimizes prediction error under sparsity constraints on the prediction coefficients. In order to solve this problem, a novel solver is introduced as an extension of LARS for multi-class support. Experiments show that the proposed scheme can reduce the amount of encoded bits by 1.21% to 3.24% on average compared to HM16.7.

Higher frame-rates are essential in achieving more realistic representations. Since increasing the frame-rate increases the total amount of information, efficient coding methods are required. However, the statistical properties of such data, needed for designing sufficiently powerful encoders, have not been clarified. Conventional studies on encoding high frame-rate sequences do not consider the effect on the encoding bit-rate of the motion blur generated by the shutter being open. When the open interval of the shutter in the image pickup apparatus increases, motion blur occurs, which is known as the integral phenomenon. The integral phenomenon changes the statistical properties of the video signal. This paper derives, for high frame-rate video, a mathematical model that quantifies the relationship between frame-rate and bit-rate; it incorporates the effect of the low-pass filtering induced by the open shutter. A coding experiment confirms the validity of the mathematical model.

We formulate the design of an optimal quantizer as an optimization problem that finds the quantization indices that minimize quantization error. As a solution of the optimization problem, an approach based on dynamic programming, which is called DP quantization, is proposed. It is observed that quantized signals do not always contain all kinds of signal values which can be represented with given bit-depth. This property is called amplitude sparseness. Because quantization is the amplitude discretization of signal value, amplitude sparseness is closely related to quantizer design. Signal values with zero frequency do not impact quantization error, so there is the potential to reduce the complexity of the optimal quantizer by not computing signal values that have zero frequency. However, conventional methods for DP quantization were not designed to consider amplitude sparseness, and so fail to reduce complexity. The proposed algorithm offers a reduced complexity optimal quantizer that minimizes quantization error while addressing amplitude sparseness. Experimental results show that the proposed algorithm can achieve complexity reduction over conventional DP quantization by 82.9 to 84.2% on average.