Perceptual Coding based on JND model for

Perceptual Coding based on JND model for High
Efficiency Video Coding
Woong Lim, Hyunho Jo and Donggyu Sim
Department of Computer Engineering
Kwangwoon University
Seoul, Korea
Abstract—In this paper, we propose a video compression algorithm
based on human visual perception for High Efficiency Video Coding
(HEVC). The proposed algorithm is applied to the conventional lossy
coding to reduce bitrate without any further subjective quality
degradation. It improves coding performance by allowing objective
quantization error which is not noticeable to human visual system
(HVS). By using luminance just noticeable distortion (JND) model,
the proposed algorithm finds out the maximum QP value based on
JND threshold to reduce bitrate and not to degrade the subjective
quality. Based on the JND threshold, the proposed algorithm quantize
the transformed residual within a certain range of quantization
parameter (QP) to minimize bitrate with retaining subjective quality
in the HVS’ perspective. We found that the proposed algorithm
shows maximum 18.37% and 8.27% in average bitrate reduction with
similar subjective quality.
Keywords-HEVC, perceptual coding, JND, Subjective quality
I.
INTRODUCTION
Recently, market demand for high resolution video services
beyond full high definition (HD) is rapidly increasing.
Especially in the multimedia services, video takes possession
of a huge amount of data and compression technologies have
been developed. To support the services effectively, Moving
Picture Experts Group (MPEG) of ISO/IEC and Video Coding
Experts Group (VCEG) of ITU-T have organized Joint
Collaborative Team on Video Coding (JCT-VC) and
standardized HEVC [1]. It is evaluated that HEVC achieves
over 50% higher compression efficiency compared to the
conventional H.264/AVC [2]. However, although the ultimate
visual quality is the subjectively perceived quality general
video codec including H.264/AVC and HEVC compress video
with rate-distortion cost (rdcost) based on the amount of bits
and objective quality e.g. Mean Squared Error (MSE) because
of the computational complexity. Therefore, it needs to
consider the subjective quality and there is a room for further
improvement of the coding performance.
In many applications e.g. video streaming on the internet or
broadcasting, lossy coding methods are widely used because of
the low bitrate for the limited channel bandwidth. In general,
the lossy coding achieves high bitrate reduction with allowance
of error caused by quantization and the strength of the
quantization is controlled by Quantization Parameter (QP). As
mentioned before, the conventional lossy coding is conducted
using Rate-Distortion Optimization (RDO) which uses an
rdcost based on the bitrate and distortion using Lagragean
multiplier to simultaneously consider both the amount of bits
and distortion. However, the objectively calculated distortion is
not always highly correlated to HVS characteristics and cannot
always guarantee the optimal visual quality. Therefore, we
propose the method to improve the coding performance
reducing the bitrate with the same perceptual quality which
controls the QP based on the characteristics of HVS by using
luminance JND model for HEVC.
The rest of this paper is organized as follows. Section II
presents the characteristics of HVS using luminance JND
model. Section III describes the detail of the proposed method
to control the QP based on the JND threshold on HEVC.
Section IV shows the experimental results and this paper is
concluded in Section V.
II. LUMINANCE ADATATION JND MODEL
The perceptual quality of video depends on HVS and the
HVS is a very complicated system to decompose each
characteristic clearly. However, there are many dominant
features of HVS e.g. contrast sensitivity, motion adaptation and
etc. and the one of those dominant characteristics is luminance
adaptation effect. The luminance adaptation effect is a key
concept of the perceptual coding and modeled as JND function
[3]. It represents the visibility of a level of error as a threshold
on the pixel domain with consideration of local background
luminance, i.e. visual sensitivity varies depending on the
background luminance intensity and the luminance adaptation
JND model defines the visible threshold of error according to
the local background luminance. The luminance adaptation
JND model in case of the bit depth equal to 8 can be defined as
(1) [4].
ì æ
I ( x, y ) ö
÷ + 3, if I ( x, y ) £ 127
ïï17çç1 (1)
127 ÷ø
JNDlum ( x, y ) = í è
ï 3 (I ( x, y ) - 127 ) + 3, otherwise
ïî 128
In (1), JNDlum indicates the threshold of luminance adaptation
JND, x and y are 2 dimensional image coordinates and I means
the local luminance average, respectively.
This luminance adaptation JND model shows that the HVS
is more sensitive at the mid-level luminance background rather
than the bright or dark region. It means that human beings can
find out the coding error at the mid-level background region
more easily. Conversely, the quantization error around the
bright or dark region is less noticeable. Therefore, we can
reduce the bitrate by applying the strong quantization to the
bright or dark region based on the luminance adaptation JND
threshold without any perceptual quality degradation. Fig. 1
shows the luminance threshold to the local average luminance
background intensity in case the bit-depth is equal to 8 based
on (1).
estimation and new motion prediction methods, motion vector
merge and Advanced Motion Vector Prediction (AMVP) for
inter prediction are adopted. For in-loop filter, Sample
Adaptive Offset (SAO) is added to the conventional deblocking
filter. Moreover, various sizes of block structure, so called
Coding Unit (CU), Prediction Unit (PU) and Transform Unit
(TU) is adopted. Fig. 2 shows the overall block diagram of
HEVC encoder. In this paper, we propose the perceptual
coding method to reduce the bitrate without degradation of
visual quality for HEVC. To achieve this goal, the proposed
method decide the QP value based on the threshold from
luminance adaptation JND model described in Section II.
Most video codecs employ the RDO to improve the coding
performance. In the RDO process, the objective error
measurement e.g. mean squared error (MSE) is used. The
conventional rdcost for RDO is described as
rdcost = D + lR .
(2)
In (2), D and R indicate distortion and bitrate, respectively.
And λ is Lagrangean multiplier. General encoding methods
employ the rdcost based on the bitrate and distortion to decide
the optimal coding mode.
Figure 1. Approximated luminance JND
Using the luminance adaptation JND function shown in Fig.
1, we can find out pixel-wise threshold of noticeable error
according to the average luminance intensity of local
background.
III.
THE PROPOSED PERCEPTUAL CODING METHOD USING
LUMINANCE ADAPTATION JND MODEL FO HEVC
Figure 2. Overall block diagram of HEVC encoder
HEVC achieves higher coding performance using various
improved coding tools compared to the conventional
H.264/AVC. More prediction directions for intra prediction are
employed, DCT-based interpolation filter for motion
However, the objective quality measurement has a
limitation that does not always reflect the subjective quality
correctly. It means that if the quantization error is not
noticeable, we can quantize the transformed coefficients
properly and it also reduces the amount of bits without
subjective quality degradation. In other words, human beings
cannot find out the difference of error according to the intensity
of the local average background luminance even if the higher
QP value is applied. It means that the higher quantization error
cannot affect the subjective visual quality in case of the local
background is very dark or bright. Using the luminance
adaptation JND threshold, we can decide the proper QP for the
perceptual coding. The overall block diagram of the proposed
algorithm is shown in Fig. 3.
Figure 3. The block diagram of the proposed method
As shown in Fig. 3, the proposed method iteratively
changes the QP for the current CU from the base QP. The
proper QP value of the current CU is decided based on the
quantization error between the original input signal and the
reconstructed signal coded by the conventional RDO prior to
in-loop filtering. In case of the quantization error which is less
than the luminance adaptation JND threshold (i.e. the
quantization error is not noticeable), the higher QP value can
be applied adaptively. After the iteration for the current CU,
the proper QP is decided and the delta QP is signaled for the
current CU. As a result, the proposed algorithm finds out the
maximum QP for the current CU to reduce the amount of
generated bits retaining the perceptual quality as is was coded
using the base QP.
IV.
PERFORMANCE EVALUATION OF THE PROPOSED
ALGORITHM
The proposed method is implemented on HM11.0 which is
the reference software of HEVC [5]. For the performance
evaluation, the experimental results of the proposed method is
compared to the coding results using Common Test Condition
(CTC) provided by JCT-VC [6] as an anchor and 2 ‘class B’
(1920×1080 resolution) test sequences, ‘Kimono’ and
‘ParkScene’, are used. The experiments of the proposed
algorithm is performed under the same encoding condition.
The only difference is that the proposed algorithm increases the
QPs for CUs from the base QP up to 90% of the pixels in a CU
do not exceed the luminance adaptation JND threshold.
Table 1 shows the comparison of coding results between
the anchor and the proposed method. As shown in Table 1, the
proposed method achieves about 8.27% in average and
maximum 18.37% bitrate reduction compared to the anchor. It
means the proposed algorithm reduces the bitrate with the same
subjective quality.
TABLE 1.
Kimono
ParkScene
Figure 4. Visual quality of (a) the anchor with fixed QP equal to 22 and (b)
the proposed method with various QPs from 22
Fig. 4 shows the visual quality of the anchor and the
proposed method with the base QP is equal to 22 for ‘Kimono’
sequence which means the QP of the anchor is fixed for whole
CUs in a slice and the QPs of the proposed algorithm starts
from the base QP and varies according to the luminance
adaptation JND threshold.
As shown in Fig. 4 and Table 1, the bitrate reduction of the
proposed algorithm is over 15% with similar visual quality.
PERFORMANCE EVALUATION OF THE PROPOSED METHOD
Anchor
Sequence
(b)
22
4735.40
Perceptual delta QP
delta Rate
(%)
3986.09
15.82
27
2164.10
1916.69
11.43
32
37
22
27
32
37
1054.08
533.55
7413.76
3179.41
1450.55
670.73
1005.63
529.56
6051.50
2840.54
1395.62
665.67
4.60
0.75
18.37
10.66
3.79
0.76
QP
bitrate
bitrate
Figure 5. Selected QPs using the proposed algorithm for ‘Kimono’ sequence
from the base QP equal to 22
Fig. 5 shows the QP selection of the proposed method. The
bright block indicates the higher QP compared to the base QP.
As shown in Fig. 5, the proposed algorithm adaptively select
the proper QPs based on the luminance adaptation JND
threshold.
V. CONCLUSION
In this paper, we proposed a perceptual video coding
algorithm based on luminance adaptation JND model for
HEVC. The proposed algorithm is implemented on the
HM11.0 reference software of HEVC. It is applied to the
conventional coding loop and decides the proper QP value for
each CU based on the luminance adaptation JND threshold to
reduce bitrate without perceptual quality degradation. By using
the proposed algorithm, we achieved 8.27% in average and
maximum 18.37% bitrate reduction with similar subjective
visual quality.
(a)
ACKNOWLEDGMENT
This research was supported by the MSIP(Ministry of
Science, ICT & Future Planning), Korea, under the
ITRC(Information Technology Research Center) support
program supervised by the NIPA(National IT Industry
Promotion Agency) (NIPA-2014-H0301-14-1018)
[3]
[4]
[5]
REFERENCES
[1]
[2]
G. J. Sullivan, J. R. Ohm, W. Han, and T. Wiegand, “Overview of the
High Efficiency Video Coding (HEVC) standard,” IEEE Trans. Circuits
Syst. Video Technol., vol. 22, pp. 1649-1668, December 2012.
A. Tamhankar and K.-R. Rao, “An Overview of H.264/MPEG-4 part
10,” 4th EURASIP Conference on Video/Image Processing and
Multimedia Communications, vol. 1, pp. 1-51, June 2003.
[6]
C. Chou, and Y. Li, “A perceptually tuned subband image coder based
on the measure of just-noticeable-distortion profile,” IEEE Trans.
Circuits Syst. Video Technol., vol. 5, pp. 467-476, December 1995.
C. Lee, P. Lin, L. Chen, and W. Wang, “Image enhancement approach
using the just-noticeable-difference model of the human visual system,”
J. Electron. Imageing., vol. 21, July 2012.
Y. Wang, G. J. Sullivan, and B. Bross, “High Efficiency Video coding
(HEVC) Defect Report 3,” Joint Collaborative Team on Video Coding
(JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,
JCTVC-P1003_v1, San José, US, January 2014.
F. Bossen, “Common test conditions and software reference
configurations,” The 8th JCT-VC meeting, JCT-VC H1100, San Jose,
CA, Jan. 2012.

Download Report