High-Level Synthesis Implementation of an Embedded Real-Time HEVC Intra Encoder on FPGA for Media Applications

Sjövall, Panu; Lemmetti, Ari; Vanne, Jarno; Lahti, Sakari; Hämäläinen, Timo D.

doi:10.1145/3491215

Cited by 9 publications

(3 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The design input of hardware also must be software instead of the hardware description language to implement different NPR hardware modules flexibly and quickly. So, we have proposed a smart glass with a field programmable gate array, FPGA, and are developing an NPR library that can be converted to efficient hardware automatically by high-level synthesis (2)(3)(4) . To our best knowledge, there is no research about such smart glass and the HLS-oriented NPR library.…”

Section: Introductionmentioning

confidence: 99%

Process Chaining Without Image and Performance Loss for Pencil Drawing Style Image Conversion in High-Level Synthesis

Tani,

Yamawaki

2023

The Proceedings of the 10th International Conference on Intelligent Systems and Image Processing 2023

View full text Add to dashboard Cite

We are developing high-performance and low-power non-photorealistic rendering hardware for an embedded image processing system using High-Level Synthesis (HLS) technology, which automatically converts software to hardware. The hardware of pencil drawing style image conversion, which is one of NPRs, has been studied as a first step. To make HLS generate a well-pipelined hardware module with burst transfer, we have applied the memory access streaming technique for window-level processing. However, this technique has a weak point that lacks the upper and left side edges of the input image while the ideal performance can be achieved. This paper develops HLSoriented software for the pencil drawing style conversion, which can compensate for the shaved image without affecting the performance. The experimental result shows that our proposal can obtain the output image without any lack. In addition, the effects lead by our extension to the execution time, hardware resource, and power efficiency are little.

show abstract

Section: Introductionmentioning

confidence: 99%

Process Chaining Without Image and Performance Loss for Pencil Drawing Style Image Conversion in High-Level Synthesis

Tani,

Yamawaki

2023

The Proceedings of the 10th International Conference on Intelligent Systems and Image Processing 2023

View full text Add to dashboard Cite

show abstract

“…Many of them include specialized hardware such as field-programmable gate arrays (FPGAs), while others are focused only on optimized algorithms at the software level. FPGAs are expensive and oriented toward companies and professionals that mostly need real-time video-encoding hardware [2][3][4][5]. In contrast, optimized software algorithms have been introduced to reduce the computational load [6][7][8] in variants of existing software video encoders without any additional costs.…”

Section: Introductionmentioning

confidence: 99%

Fraction Execution Resolver Using a Hybrid Multi-CPU/GPU Encoding Scheme

Papaioannou,

Koziri,

Loukopoulos

et al. 2023

Electronics

View full text Add to dashboard Cite

Modern video coding standards make use of sub-pixel motion estimation to improve the video quality and reduce the bitrate. It is known that the fraction motion estimation (FME) part follows the integer motion estimation (IME) and adds an extra computational overhead due to the interpolation and the additional motion searches. In this paper, we propose a fraction execution resolver (FER) algorithm that lets the encoder skip the fraction part when specific criteria are met by introducing a preliminary fast test decision point (pFTDP) function for the IME part. If the pFTDP returns zero motion vectors (MVs) and the displacement search area center is also zero, then the fraction part is skipped. The pFTDP decision maker is executed only once, when a 2N × 2N block is first met, while all subsequent blocks follow this initial decision either by receiving the necessary MVs and RD from the pFTDP function or by using the precalculated IME values from the GPU kernel. For our experiments, we use a multithreaded CPU environment that also makes use of GPUs only for the integer part. Our evaluations provide a greater than 1600% encoding time saving at its peak in comparison with the default HEVC sequential mode and ideally a saving of greater than 2286% for still video frame sequences. The total average speedup for both Class A and Class B video sequences is ×13.45. The gain of the FER itself is more than ×3.9 compared with the same multithreaded setup environment. The PSNR and bitrate overhead observed are proportional to the tiling scheme used and are more related to the way CABAC works internally. The FER’s negative effects on coding efficiency are proven to be negligible. A balance between speed and quality achieved by using a lower tiling pattern is shown to minimize the negative effects of the encoding scheme pattern. The experimental results confirm the validity of our motivation, namely, that we can benefit from a software fraction execution resolver without any extra hardware costs. The gain is further increased when video sequences have more static blocks than others.

show abstract

“…Cai et al [11] proposed an efficient intra mode decision algorithm for the parallel hardware architecture of the AVS3 intra encoder by processing CUs in parallel, including intra prediction and estimating the rate-distortion cost for mode decision. Sjövall et al [12] introduced the first complete high-level synthesis (HLS) implementation for an HEVC intra encoder on FPGA, and designed a proof-of-concept system for hardwareaccelerated HEVC encoding. Zummach et al [13] proposed a hardware design for the AV1-constrained directional enhancement filter, targeting the real-time processing of 4K ultra-high-definition videos.…”

Section: Introductionmentioning

confidence: 99%

Bagged Tree and ResNet-Based Joint End-to-End Fast CTU Partition Decision Algorithm for Video Intra Coding

Peng

et al. 2022

Electronics

View full text Add to dashboard Cite

Video coding standards, such as high-efficiency video coding (HEVC), versatile video coding (VVC), and AOMedia video 2 (AV2), achieve an optimal encoding performance by traversing all possible combinations of coding unit (CU) partition and selecting the combination with the minimum coding cost. It is still necessary to further reduce the encoding time of HEVC, because HEVC is one of the most widely used coding standards. In HEVC, the process of searching for the best performance is the source of most of the encoding complexity. To reduce the complexity of the coding block partition in HEVC, a new end-to-end fast algorithm is presented to aid the partition structure decisions of the coding tree unit (CTU) in intra coding. In the proposed method, the partition structure decision problem of a CTU is solved by a novel two-stage strategy. In the first stage, a bagged tree model is employed to predict the splitting of a CTU. In the second stage, the partition problem of a 32 × 32-sized CU is modeled as a 17-output classification task for the first time, so that it can be solved by a single prediction. To achieve a high prediction accuracy, a residual network (ResNet) with 34 layers is employed. Jointly using bagged tree and ResNet, the proposed fast CTU partition algorithm is able to generate the partition quad-tree structure of a CTU through an end-to-end prediction process, which abandons the traditional scheme of making multiple decisions at various depth levels. In addition, several datasets are used in this paper to lay the foundation for high prediction accuracy. Compared with the original HM16.7 encoder, the experimental results show that the proposed algorithm can reduce the encoding time by 60.29% on average, while the Bjøntegaard delta rate (BD-rate) loss is as low as 2.03%, which outperforms the results of most of the state-of-the-art approaches in the field of fast intra CU partition.

show abstract

High-Level Synthesis Implementation of an Embedded Real-Time HEVC Intra Encoder on FPGA for Media Applications

Cited by 9 publications

References 36 publications

Process Chaining Without Image and Performance Loss for Pencil Drawing Style Image Conversion in High-Level Synthesis

Process Chaining Without Image and Performance Loss for Pencil Drawing Style Image Conversion in High-Level Synthesis

Fraction Execution Resolver Using a Hybrid Multi-CPU/GPU Encoding Scheme

Bagged Tree and ResNet-Based Joint End-to-End Fast CTU Partition Decision Algorithm for Video Intra Coding

Contact Info

Product

Resources

About