ApproxHPVM: a portable compiler IR for accuracy-aware optimizations

Sharif, Hashim; Srivastava, Prakalp; Huzaifa, Muhammad; Kotsifakou, Maria; Joshi, Keyur D.; Sarita, Yasmin; Zhao, Nathan; Adve, Vikram; Misailovíc, Saša; Adve, Sarita V.

doi:10.1145/3360612

Cited by 16 publications

(8 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ApproxHPVM [39] expands HPVM by introducing the support for tensor operations commonly used in NNs: multiplication, convolution, addition, pooling, and activation functions. Additionally, ApproxHPVM enables transforming high level descriptions of convolutional neural networks (in frameworks such as Keras, PyTorch) into DFGs in the form of generated HPVM-C source files.…”

Section: Preliminariesmentioning

confidence: 99%

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Fabjančič¹,

Machidon²,

Sharif³

et al. 2023

Preprint

View full text Add to dashboard Cite

Runtime-tunable context-dependent network compression would make mobile deep learning adaptable to often varying resource availability, input "difficulty", or user needs. The existing compression techniques significantly reduce the memory, processing, and energy tax of deep learning, yet, the resulting models tend to be permanently impaired, sacrificing the inference power for reduced resource usage. The existing tunable compression approaches, on the other hand, require expensive re-training, seldom provide mobile-ready implementations, and do not support arbitrary strategies for adapting the compression.In this paper we present Mobiprox, a framework enabling flexible-accuracy on-device deep learning. Mobiprox implements tunable approximations of tensor operations and enables runtime adaptation of individual network layers. A profiler and a tuner included with Mobiprox identify the most promising neural network approximation configurations leading to the desired inference quality with the minimal use of resources. Furthermore, we develop control strategies that depending on contextual factors, such as the input data difficulty, dynamically adjust the approximation level of a model. We implement Mobiprox in Android OS and through experiments in diverse mobile domains, including human activity recognition and spoken keyword detection, demonstrate that it can save up to 15% system-wide energy with a minimal impact on the inference accuracy.CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing; • Computing methodologies → Neural networks.

show abstract

Section: Preliminariesmentioning

confidence: 99%

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Fabjančič¹,

Machidon²,

Sharif³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In approximate computing the aim is to create a approximate version of a calculation (surrogate) and dynamically substitute it, where appropriate, in place of the exact or "golden" calculation. Typically, approximate computing deals with functions within a single HPC application, replacing them with optimized variants at the cost of accuracy [28,30,32,39,46,48]. In general, it aims to address three major technical challenges: a) identifying calculations for which to create surrogates [30,48]; b) generating well-performing surrogates [32,39]; and c) deciding when to replace a calculation with one of its surrogates (adjudication) [28,46].…”

Section: Approximation In Computational Sciencementioning

confidence: 99%

“…Typically, approximate computing deals with functions within a single HPC application, replacing them with optimized variants at the cost of accuracy [28,30,32,39,46,48]. In general, it aims to address three major technical challenges: a) identifying calculations for which to create surrogates [30,48]; b) generating well-performing surrogates [32,39]; and c) deciding when to replace a calculation with one of its surrogates (adjudication) [28,46]. The surrogate methods may be also be supported by, or implemented on, specialized hardware accelerators.…”

Section: Approximation In Computational Sciencementioning

confidence: 99%

Towards an Approximation-Aware Computational Workflow Framework for Accelerating Large-Scale Discovery Tasks

Johnston¹,

Vassiliadis²

2022

Preprint

View full text Add to dashboard Cite

The use of approximation is fundamental in computational science. Almost all computational methods adopt approximations in some form in order to obtain a favourable cost/accuracy trade-off and there are usually many approximations that could be used. As a result, when a researcher wishes to measure a property of a system with a computational technique, they are faced with an array of options. Current computational workflow frameworks focus on helping researchers automate a sequence of steps on a particular platform. The aim is often to obtain a computational measurement of a property. However these frameworks are unaware that there may be a large number of ways to do so. As such, they cannot support researchers in making these choices during development or at execution-time.We argue that computational workflow frameworks should be designed to be approximation-aware -that is, support the fact that a given workflow description represents a task that could be performed in different ways. This is key to unlocking the potential of computational workflows to accelerate discovery tasks, particularly those involving searches of large entity spaces. It will enable efficiently obtaining measurements of entity properties, given a set of constraints, by directly leveraging the space of choices available. In this paper we describe the basic functions that an approximationaware workflow framework should provide, how those functions can be realized in practice, and illustrate some of the powerful capabilities it would enable, including approximate memoization, surrogate model support, and automated workflow composition.

show abstract

“…[87], and Flex-Tensor [88] optimize algorithms and task allocations for network traffic in data-center-scale edge computing or single-server computing to reduce infrastructure costs. Language frameworks like ApproxHPVM [89] and ApproxTuner [90] further helps programmer to estimate and optimize the loss of accruacy in ML workloads. The GPETPU framework is orthogonal to the aforementioned research because GPETPU is compatible with existing heterogeneous computing platforms; Edge TPUs can function as complementary hardware accelerators within the system.…”

Section: Related Workmentioning

confidence: 99%

GPTPU: Accelerating Applications using Edge Tensor Processing Units

Hsu¹,

Tseng²

2021

Preprint

View full text Add to dashboard Cite

Neural network (NN) accelerators have been integrated into a widespectrum of computer systems to accommodate the rapidly growing demands for artificial intelligence (AI) and machine learning (ML) applications. NN accelerators share the idea of providing native hardware support for operations on multidimensional tensor data. Therefore, NN accelerators are theoretically tensor processors that can improve system performance for any problem that uses tensors as inputs/outputs. Unfortunately, commercially available NN accelerators only expose computation capabilities through AI/MLspecific interfaces. Furthermore, NN accelerators reveal very few hardware design details, so applications cannot easily leverage the tensor operations NN accelerators provide.This paper introduces General-Purpose Computing on Edge Tensor Processing Units (GPETPU), an open-source, open-architecture framework that allows the developer and research communities to discover opportunities that NN accelerators enable for applications. GPETPU includes a powerful programming interface with efficient runtime system-level support-similar to that of CUDA/OpenCL in GPGPU computing-to bridge the gap between application demands and mismatched hardware/software interfaces.We built GPETPU machine uses Edge Tensor Processing Units (Edge TPUs), which are widely available and representative of many commercial NN accelerators. We identified several novel use cases and revisited the algorithms. By leveraging the underlying Edge TPUs to perform tensor-algorithm-based compute kernels, our results reveal that GPETPU can achieve a 2.46× speedup over highend CPUs and reduce energy consumption by 40%.

show abstract

ApproxHPVM: a portable compiler IR for accuracy-aware optimizations

Cited by 16 publications

References 47 publications

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Towards an Approximation-Aware Computational Workflow Framework for Accelerating Large-Scale Discovery Tasks

GPTPU: Accelerating Applications using Edge Tensor Processing Units

Contact Info

Product

Resources

About