Rim

Abstract

Video cameras are among the most ubiquitous sensors in the Internetof-Things. Video and audio applications, such as cross-camera activity detection, avatar extraction or language translation will, in the future, offload processing to an edge cluster of GPUs. Rim is a management system for such clusters that satisfies throughput and latency requirements of these applications, while enabling high cluster utilization. It uses coarse-grained knowledge of application structure to profile throughput of applications on resources, then uses these profiles to place applications on cluster nodes to achieve these goals. It dynamically adapts placement to load and failures. Experiments show that on maximal workloads on a testbed, Rim can satisfy requirements of all applications, but competing approaches designed for low-latency GPU execution cannot. CCS Concepts• General and reference → Performance; • Computing methodologies → Neural networks; Computer vision; Natural language processing; • Software and its engineering → Scheduling.

Edge AI: A survey

“…As of 2019, the worldwide edge computing industry is estimated to be worth $3.5 billion. By 2027, that figure might have increased to $43.4 billion [74]. This suggests that AI computation on such devices will grow at an exponential rate, allowing consumers to engage with compiled code at the source in a safe and optimal manner [75].…”