FAIRSEQ is an open-source sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling, and other text generation tasks. The toolkit is based on PyTorch and supports distributed training across multiple GPUs and machines. We also support fast mixed-precision training and inference on modern GPUs. A demo video can be found here: https://www.youtube. com/watch?v=OtgDdWtHvto.

Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.

No abstract

objects at multiple scales, in context and among clutter, and under frequent occlusion. Moreover, the COCO evaluation metric rewards high quality localization. To addresses this, we propose the MultiPath network pictured above, which contains three key modifications: skip connections, foveal regions, and and an integral loss. Together these modifications allow information to flow along multiple paths through the network, enabling the classifier to operate at multiple scales, utilize context effectively, and perform more precise object localization. Our MultiPath network, coupled with DeepMask object proposals [4, 5], achieves major gains on COCO detection.The recent COCO dataset presents several new challenges for object detection. In particular, it contains objects at a broad range of scales, less prototypical images, and requires more precise localization. To address these challenges, we test three modifications to the standard Fast R-CNN object detector: (1) skip connections that give the detector access to features at multiple network layers, (2) a foveal structure to exploit object context at multiple object resolutions, and (3) an integral loss function and corresponding network adjustment that improve localization.The result of these modifications is that information can flow along multiple paths in our network, including through features from multiple network layers and from multiple object views. We refer to our modified classifier as a 'MultiPath' network. We couple our MultiPath network with DeepMask object proposals, which are well suited for localization and small objects, and adapt our pipeline to predict segmentation masks in addition to bounding boxes. The combined system improves results over the baseline Fast R-CNN detector with Sel-Search by 66% overall and by 4× on small objects.

In compressed sensing, one takes n < N samples of an N-dimensional vector x 0 using an n × N matrix A, obtaining undersampled measurements y = Ax 0 . For random matrices with independent standard Gaussian entries, it is known that, when x 0 is k-sparse, there is a precisely determined phase transition: for a certain region in the (k/n,n/N)-phase diagram, convex optimization min || x || 1 subject to y = Ax, x ∈ X N typically finds the sparsest solution, whereas outside that region, it typically fails. It has been shown empirically that the same property-with the same phase transition location-holds for a wide range of non-Gaussian random matrix ensembles. We report extensive experiments showing that the Gaussian phase transition also describes numerous deterministic matrices, including Spikes and Sines, Spikes and Noiselets, Paley Frames, Delsarte-Goethals Frames, Chirp Sensing Matrices, and Grassmannian Frames. Namely, for each of these deterministic matrices in turn, for a typical k-sparse object, we observe that convex optimization is successful over a region of the phase diagram that coincides with the region known for Gaussian random matrices. Our experiments considered coefficients constrained to X N for four different sets X ∈ {[0, 1], R + , R, C}, and the results establish our finding for each of the four associated phase transitions.sparse recovery | universality in random matrix theory equiangular tight frames | restricted isometry property | coherence C ompressed sensing aims to recover a sparse vector x 0 ∈ X N from indirect measurements y = Ax 0 ∈ X n with n < N, and therefore, the system of equations y = Ax 0 is underdetermined. Nevertheless, it has been shown that, under conditions on the sparsity of x 0 , by using a random measurement matrix A with Gaussian i.i.d entries and a nonlinear reconstruction technique based on convex optimization, one can, with high probability, exactly recover x 0 (1, 2). The cleanest expression of this phenomenon is visible in the large n; N asymptotic regime. We suppose that the object x 0 is k-sparse-has, at most, k nonzero entries-and consider the situation where k ∼ ρn and n ∼ δN. Fig. 1A depicts the phase diagram ðρ; δ; Þ ∈ ð0; 1Þ 2 and a curve ρ*ðδÞ separating a success phase from a failure phase. Namely, if ρ < ρ*ðδÞ, then with overwhelming probability for large N, convex optimization will recover x 0 exactly; however, if ρ > ρ*ðδÞ, then with overwhelming probability for large N convex optimization will fail. [Indeed, Fig. 1 depicts four curves ρ*ðδjXÞ of this kind for X ∈ f½0; 1; R + ; R; Cg-one for each of the different types of assumptions that we can make about the entries of x 0 ∈ X N (details below).]How special are Gaussian matrices to the above results? It was shown, first empirically in ref. 3 and recently, theoretically in ref. 4, that a wide range of random matrix ensembles exhibits precisely the same behavior, by which we mean the same phenomenon of separation into success and failure phases with the same phase boundary. Such universality, if exhib...