Flux: Elegant machine learning with Julia

Abstract

SummaryFlux is library for machine learning (ML), written using the numerical computing language Julia (Bezanson et al. 2017). The package allows models to be written using Julia's simple mathematical syntax, and applies automatic differentiation (AD) to seamlessly calculate derivatives and train the model. Meanwhile, it makes heavy use of Julia's language and compiler features to carry out code analysis and make optimisations. For example, Julia's GPU compilation support (Besard, Foket, and De Sutter 2017) can be used to JIT-compile custom GPU kernels for model layers (Innes and others 2017a).The machine learning community has traditionally been divided between "static" and "dynamic" frameworks that are easy to optimise and easy to use, respectively (Innes and others 2017b). Flux blurs the line between these two approaches, combining a highly intuitive programming model with the compiler techniques needed by ML. This enables research into advanced compiler transforms such as batching (Bradbury 2018) without changing any user code.Flux has been used heavily for natural language processing, but can also support state-ofthe-art research models in areas like computer vision, reinforcement learning and robotics. Many examples of such models can be found in the model zoo (Innes and others 2018).

Embeddings.jl: easy access to pretrained word embeddings from Julia

“…Embeddings.jl exposes these as a standard matrix of numbers and a corresponding array of strings. This lets Julia programs use word embeddings easily, either on their own or alongside machine learning packages such as Flux (Innes, 2018). In such deep learning packages, it is common to use word embeddings as an input layer of a LSTM (long short term memory) network or other machine learning model, where they may be kept invariant or used as initialization for fine-tuning on the supervised task.…”

MLJ: A Julia package for composable machine learning

“…Another major advantage of a single-language solution is the ability to automatically differentiate (AD) functions from their code representations. The Flux.jl package (Innes, 2018), for example, already makes use of AD to allow unparalleled flexibility in neural network design.…”

Cooperation-Aware Reinforcement Learning for Merging in Dense Traffic

“…The parameters of the DQN algorithm are summarized in table I. It was implemented using the Flux.jl library [20].…”