Tools adopted in HM0.9 include 12-tap DCT-based interpolation filter (high efficiency configuration) and 6-tap directional interpolation filter (low complexity configuration) for 1/4 luma sample, bilinear interpolation filter for 1/8 chroma sample (both HE and LC), advanced motion vector prediction, bi-direction rounding control, bi-directional prediction for temporal level 0.

Motion compensation is the key factor for high efficient video compressing, where fractional pel accuracy requir[......]

Read More...

In current HM, unified intra prediction provides up to 34 directional prediction modes for different PUs. With the PU size of 4×4, 8×8, 16×16, 32×32, 64×64, there are 17, 34, 34, 34, and 5 prediction modes available respectively. The prediction directions in the unified intra prediction have the angles of +/- [0, 2, 5, 9, 13, 17, 21, 26, 32]/32. The angle is given by displacement of the bottom row of the PU and the reference row above the PU in case of vertical prediction, or displacement of the rightmost column of the PU and the reference column left from the PU in case of horizontal prediction. Figure 1 shows an example of prediction directions for 32×32 block size. Instead of different accuracy for different sizes, the reconstruction of the pixel uses the linear interpolation of the reference top or left samples at 1/32th pixel accuracy for all block size.

Figure 1. Available prediction directions in the unified intra prediction

Two arrays of reference samples are used in the unified intra prediction, corresponding to the row of samples lying above the current PU to be predicted, and the column of samples lying to the left of the same PU. Given a dominant prediction direction (horizontal or vertical), one of the reference arrays is defined to be the main array and the other array the side array. In the case of vertical prediction, the reference row above the PU is called the main array and the reference column to the left of the same PU is called the side array. In the case of horizontal prediction, the reference column to the left of the PU is called the main array and the reference row above the PU is called the side array.

When the intra prediction angle is positive, blue lines in Figure 1, only the samples from the main array are used for prediction. When the intra prediction angle is negative, red lines in Figure 1, a per-sample test should be performed to determine whether samples from the main or the side array should be used for prediction, as shown in Figure 2 (a). Additionally when the side array is used, the computation of the index into the side array requires a division operation. In order to remove the division operation, the lookup-table (LUT) technique is used for the negative angular prediction process in the calculation of the y-intercept in the case of vertical prediction or the x-intercept in the case of horizontal prediction. Normally, the integer and fractional parts of the intercept between are calculated using the following equations respectively:

deltaIntSide = (256*32*(l+1)/absAng)>>8

deltaFractSide = (256*32*(l+1)/absAng)%256

where absAng is the absolute value of intra prediction angle (=[ 2,    5,   9,  13,  17,  21,  26,  32]) and l is x/y pixel location for vertical/horizontal prediction. With LUT technique, the above equations can be replaced with the following equations:

deltaIntSide = (invAbsAngTable[absAngIndex]*(l+1))>>8

deltaFractSide = (invAbsAngTable[absAngIndex]*(l+1))%256

where invAbsAngTable = [4096, 1638, 910, 630, 482, 390, 315, 256]. By using LUT, there is no division operation during the computation of the index into the side array. However, there still exists the per-sample test to determine the main or the side array for prediction. To simplify this process, the main array is extended by projecting samples from the side array onto it according to the prediction direction, as shown in Figure 2 (b). During the projection, the fractional part of the intercept is omitted and the intercept is rounded to the nearest integer:

deltaInteger = (invAbsAngTable[absAngIndex]*(l+1)+128)>>8

Finally, the prediction process only uses the extended main array and the same simple linear interpolation formula to predict all samples in the PU. Figure 2 shows an example of simplified intra prediction with direction angle VER-8.

Figure 2. An example of simplified intra prediction with direction angle VER-8

When DC mode is used in the intra prediction, the mean value of samples from both top row and left column is used for the DC prediction.

Table 1 summarizes the physical and logical mode indexes used in the bitstream and prediction functions respectively. Depending on the PU size, different numbers of prediction modes are available respectively, as listed in Table 2.

 Table 1. Physical and logical mode indexes used in the bitstream and prediction functions respectively

[......]

Read More...

CU, the basic coding unit like the macroblock and sub-macroblock in H.264/AVC, can have various sizes but is restricted to be a square shape. Given the size and hierarchical depth of LCU, CU can be expressed in a recursive quadtree representation adapted to the picture. Figure 1 (b) shows maximum possible recursive CU structure in HM (LCU size = 64 and hierarchical depth = 4). In HM, the coding unit tree structure is limited from 8×8 to 64×[......]

Read More...

Table 1. Structure of tools forming the high efficiency and low complexity configurations of the HM

Read More...

The deblocking loop filter, inherited from H.264/AVC, alleviates the blocking artifacts caused by the block-based DCT+MCP video coding framework. It uses a bank of low-pass filters, which are adaptively applied to block boundaries according to the boundary strength (BS), and provides better visual quality and improved capability to predict other pictures.

Adaptive loop filter (ALF; click here for introduction) is placed in the MCP loop after the deblocking process, and is used to restore the degraded picture (caused by compression) such that the MSE between the reconstructed and source frames is minimized. The coefficients of ALF are calculated and transmitted on a frame basis and the minimum mean squared error (MMSE) estimator is used. For each degraded frame, ALF can be applied to the entire frame or to local areas. The former is known as frame-based ALF. In the latter case, additiona[......]

Read More...

Motion-compensated prediction (MCP) is the key to the success of the modern video coding standards, as it removes the temporal redundancy in video signals and reduces the size of bitstreams significantly. With MCP, the pixels to be coded are predicted from the temporally neighboring ones, and only the prediction errors and the motion vectors (MV) are transmitted. However, due to the finite sampling rate, the actual position of the prediction in the neighboring frames may be out of the sampling grid, where the intensity is unknown, so the intensities of the positions in between the integer pixels, called sub-positions, must be interpolated and the resolution of MV is increased accordingly.  

Interpolation in H.264/AVC  

In H.264/AVC, for the resolution of MV is quarter-pixel, the reference frame is interpolated to be 16 times the size for MCP, 4 times both sides. As shown in Fig. 1(a), the interpolation defined in H.264 includes two stages, inter[......]

Read More...

TMuC provides more flexibility than H.264/AVC. The  basic coding unit, called coding tree block (CTB), which has a similar role to the macroblocks in H.264/AVC, can have variable sizes (a power of 2). The sizes of the largest and smallest CTBs are specified in the sequence parameter set (SPS). A frame is divided into non-overlapped largest CTBs (LCTB), e.g., 128×128, and then each LCTB can be further divided in a recursive tree representation.

Each CTB has its own prediction type (intra/inter) and prediction partition. The partition can be symmetric, just as in H.264/AVC, or asymmetric, e.g., 64×64 block can be partitioned into 64×16/64×48 or 16×64/48&[......]

Read More...

Mode-dependent directional transform (MDDT) was proposed to compact the residue produced by intra prediction. It consists of a series of pre-defined separable transforms; each transform is efficient in compacting energy along one of the prediction directions, thus favoring one of the intra modes. The type of MDDT is coupled with the selected[......]

Read More...

As far as adaptive loop filter (ALF) is concerned, there are three types of ALF: frame-based, block-based and quadtree-based ALFs. All of them are based on wiener filter, but with different filtering control basis. In frame-based ALF [VCEG-C437/AI14, C402], only one picture level flag is used to signal the decision of filtering or non-filtering.

Although wiener filter can restore the reconstructed picture to the original picture globally, there are degraded pixels locally. Since the degraded area reduce the filtering efficiency, if these areas are not filtered, the capabilities of picture restoration and loop filtering are improved. Therefore, block-based ALF [VCEG-AI18/AJ13] use explicit flags for filtering on-off on block by block basis, while quadtree-based ALF [VCEG-C181/AK22] introduces a quadtree data structure to carry out the variable-size block filtering.

3.1 Block-based Adaptive Loop Filter

Block-based ALF is an improvement of frame-based ALF. Figure 2[......]

Read More...

The basic idea of adaptive post/loop filter is the same. Both of them use adaptive wiener filtering technique to improve the quality of reconstructed picture which is degraded by compression. The difference between them is whether the filtering process is applied in or out of the core coding loop, as shown in Figure 1,  to improve the quality of reconstructed picture or just displayed picture.

Figure 1. Block diagram of JM/KTA

2. Adaptive Post Filter

In H.264/AVC, there is already an existing post-filter hint SEI message [JVT-S030/T039/U035] which provides the coefficients of a post-filter or correlation information for the design of a post-filter for potential use in post-processing of the output decoded pictures to obtain improved displayed quality.

To find the coefficients of adaptive wiener filter, the following cost function based on the whole frame is minimized:

(1)

where R is the reconstructed picture, R’ is the filtered picture, and I is the original pic[......]

Read More...