AIR: Iterative refinement acceleration using arbitrary dynamic precision

Lee, Junkyu; Peterson, Gregory D.; Nikolopoulos, Dimitrios S.; Vandierendonck, Hans

doi:10.1016/j.parco.2020.102663

Cited by 8 publications

(7 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dynamic precision approaches were also proposed for scientific computation as well as other signal processing applications to improve energy efficiency [36][37][38][39]. Lee et al [38] proposed a dynamic precision approach for linear solvers to improve energy efficiency.…”

Section: Dynamic Precision Scalingmentioning

confidence: 99%

Increased Leverage of Transprecision Computing for Machine Vision Applications at the Edge

Minhas

Lee

Mukhanov

et al. 2022

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

The practical deployment of machine vision presents particular challenges for resource constrained edge devices. With a clear need to execute multiple tasks with variable workloads, there is a need for a robust approach that can dynamically adapt at runtime and which can maintain the maximum quality of service (QoS) within the available resource constraints. A lightweight approach that monitors the runtime workload constraints and leverages accuracy-throughput trade-offs on a graphics processing unit (GPU), is presented. It includes optimisation techniques that identify the configurations for each task in terms of optimal accuracy, energy and memory and management of the transparent switching between configurations. Using a neural network architecture search that statically generates a range of implementations that target a resource-precision trade-off, we explore the detection of the optimal parameters for the required QoS under specific memory and energy constraints. For an accuracy loss of 1%, we demonstrate that a $$1.6\times$$ 1.6 × higher frame processing rate can be achieved on GPU with further improvements possible at further relaxed accuracy. In order to further improve the switching between configurations, we enhance the proposed mechanism by employing central processing units (CPUs) for offloading some of the executed frames, which helps to improve the frame rate by further 0.9%.

show abstract

Section: Dynamic Precision Scalingmentioning

confidence: 99%

Increased Leverage of Transprecision Computing for Machine Vision Applications at the Edge

Minhas

Lee

Mukhanov

et al. 2022

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

show abstract

“…Transprecise techniques used in [11], [12] adapts precision arithmetic dynamically according to runtime information such as convergence rate and numerical stability of individual computation modules. The transprecise techniques saved energy and accelerated computation without losing accuracy for linear system solvers in [11], [12].…”

Section: Related Workmentioning

confidence: 99%

TOD: Transprecise Object Detection to Maximise Real-Time Accuracy on the Edge

Lee

Varghese

Woods

et al. 2021

2021 IEEE 5th International Conference on Fog and Edge Computing (ICFEC)

Self Cite

View full text Add to dashboard Cite

Real-time video analytics on the edge is challenging as the computationally constrained resources typically cannot analyse video streams at full fidelity and frame rate, which results in loss of accuracy. This paper proposes a Transprecise Object Detector (TOD) which maximises the real-time object detection accuracy on an edge device by selecting an appropriate Deep Neural Network (DNN) on the fly with negligible computational overhead. TOD makes two key contributions over the state of the art: (1) TOD leverages characteristics of the video stream such as object size and speed of movement to identify networks with high prediction accuracy for the current frames; (2) it selects the best-performing network based on projected accuracy and computational demand using an effective and low-overhead decision mechanism. Experimental evaluation on a Jetson Nano demonstrates that TOD improves the average object detection precision by 34.7% over the YOLOv4-tiny-288 model on average over the MOT17Det dataset. In the MOT17-05 test dataset, TOD utilises only 45.1% of GPU resource and 62.7% of the GPU board power without losing accuracy, compared to YOLOv4-416 model. We expect that TOD will maximise the application of edge devices to real-time object detection, since TOD maximises real-time object detection accuracy given edge devices according to dynamic input features without increasing inference latency in practice.

show abstract

“…The AIR approach [6] uses a similar approach based on software functions which do not generate code at run time.…”

Section: Code Specializationmentioning

confidence: 99%

“…Several articles present work around transprecision computing, such as FlexFloat [5], AIR [6] or the work in [15]. For FlexFloat, Grosser et al describe the C/C++ library for transprecision computing.…”

Section: Transprecision and Challenges For Compilersmentioning

confidence: 99%

Dynamic Compilation for Transprecision Applications on Heterogeneous Platform

Dumas

Charles

Mambu

et al. 2021

JLPEA

View full text Add to dashboard Cite

This article describes a software environment called HybroGen, which helps to experiment binary code generation at run time. As computing architectures are getting more complex, the application performance is becoming data-dependent. The proposed experimental platform is helpful in programming applications that can be reconfigured at run time in order to be adapted for a new data environment. The HybroGen platform is adapted to heterogeneous architectures and can generate instructions for different targets. This platform allows to go farther than classical JIT compilation in many directions: the code generator is smaller by three orders of magnitude and faster by three orders of magnitude, compared to JIT (Just-In-Time) platforms, and allows making code transformation that is impossible in traditional compilation schemes, such as code generation for non von Neumann accelerators or dynamic code transformations for transprecision. The latter is illustrated in a code example: the square root with Newton’s algorithm. We also illustrate the proposed HybroGen platform with two other examples: a multiplication with a specialization on a value determined at run time, and a conversion of degrees Celsius to degrees Fahrenheit. This article presents a proof of concept of the proposed HybroGen platform in terms of its functionalities, and demonstrates the working status.

show abstract

AIR: Iterative refinement acceleration using arbitrary dynamic precision

Cited by 8 publications

References 27 publications

Increased Leverage of Transprecision Computing for Machine Vision Applications at the Edge

Increased Leverage of Transprecision Computing for Machine Vision Applications at the Edge

TOD: Transprecise Object Detection to Maximise Real-Time Accuracy on the Edge

Dynamic Compilation for Transprecision Applications on Heterogeneous Platform

Contact Info

Product

Resources

About