Relay: a new IR for machine learning frameworks

Abstract

Machine learning powers diverse services in industry including search, translation, recommendation systems, and security. The scale and importance of these models require that they be efficient, expressive, and portable across an array of heterogeneous hardware devices. These constraints are often at odds; in order to better accommodate them we propose a new high-level intermediate representation (IR) called Relay. Relay is being designed as a purely-functional, statically-typed language with the goal of balancing efficient compilation, expressiveness, and portability. We discuss the goals of Relay and highlight its important design constraints. Our prototype is part of the open source NNVM compiler framework, which powers Amazon's deep learning framework MxNet.

Matrix-DSP back-end support based on TVM compilation structure

“…The deep learning [10] [2] IR) resides in the front-end, and the low-level intermediate representation (tensor IR) resides in the back-end, but is relatively traditional. TVM can better obtain the overall information of the application and complete specific optimizations (such as graph optimization) for deep learning.…”

SOL: Effortless Device Support for AI Frameworks without Source Code Changes

“…TVM [21] is an optimizing compiler architecture with a large open source community and probably the highest number of supported hardware architectures. It features integration for TensorFlow and PyTorch, but so far only supports inference workloads (their high level IR "Relay" already supports training but not the lower level implementations).…”

Enumerating Hardware-Software Splits with Program Rewriting

“…To explore hardware-software splits, we begin with ML workloads written in Relay, which is the intermediate representation used by the TVM compiler [1]. Relay represents a machine learning workload as a series of kernel calls, but does not make explicit the underlying hardware and software components described above.…”