Searching Toward Pareto-Optimal Device-Aware Neural Architectures

Cheng, An-Chieh; Dong, Jin‐Dong; Hsu, Chi-Hung; Chang, Shu-Huan; Sun, Min; Chang, Shih-Chieh; Pan, Jia-Yu; Chen, Yuting; Wei, Wei; Juan, Da-Cheng

doi:10.48550/arxiv.1808.09830

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(1) Analyze the multi-objective pareto front of the designed model along the dimension of accuracy and resource footprint. Following [19] some commonly used metrics include,…”

Section: New Metricsmentioning

confidence: 99%

See 1 more Smart Citation

On-Device Machine Learning: An Algorithms and Learning Theory Perspective

Dhar,

Guo,

Liu

et al. 2019

Preprint

View full text Add to dashboard Cite

The current paradigm for using machine learning models on a device is to train a model in the cloud and perform inference using the trained model on the device. However, with the increasing number of smart devices and improved hardware, there is interest in performing model training on the device. Given this surge in interest, a comprehensive survey of the field from an device-agnostic perspective sets the stage for both understanding the state-of-the-art and for identifying open challenges and future avenues of research. Since on-device learning is an expansive field with connections to a large number of related topics in AI and machine learning (including online learning, model adaptation, one/few-shot learning, etc), covering such a large number of topics in a single survey is impractical. Instead, this survey finds a middle ground by reformulating the problem of on-device learning as resource constrained learning where the resources are compute and memory. This reformulation allows tools, techniques, and algorithms from a wide variety of research areas to be compared equitably. In addition to summarizing the state of the art, the survey also identifies a number of challenges and next steps for both the algorithmic and theoretical aspects of on-device learning.

show abstract

“…(1) Analyze the multi-objective pareto front of the designed model along the dimension of accuracy and resource footprint. Following [19] some commonly used metrics include,…”

Section: New Metricsmentioning

confidence: 99%

“…Note that, in addition to using the metrics in eq. ( 1) to ( 5) to evaluate the model performance, the works in [19,50,124] also utilize these metrics to build and optimize their learning algorithms.…”

Section: New Metricsmentioning

confidence: 99%