As neural network model sizes have dramatically increased, so has the interest in various techniques to reduce their parameter counts and accelerate their execution. An active area of research in this field is sparsity -encouraging zero values in parameters that can then be discarded from storage or computations. While most research focuses on high levels of sparsity, there are challenges in universally maintaining model accuracy as well as achieving significant speedups over modern matrix-math hardware. To make sparsity adoption practical, the NVIDIA Ampere GPU architecture introduces sparsity support in its matrix-math units, Tensor Cores. We present the design and behavior of Sparse Tensor Cores, which exploit a 2:4 (50%) sparsity pattern that leads to twice the math throughput of dense matrix units. We also describe a simple workflow for training networks that both satisfy 2:4 sparsity pattern requirements and maintain accuracy, verifying it on a wide range of common tasks and model architectures. This workflow makes it easy to prepare accurate models for efficient deployment on Sparse Tensor Cores.
An entropy balance equation including radiative heating is developed to examine the principle of minimum entropy exchange hypothesized by Paltridge. It is shown that the thermodynamic dissipation (entropy production) due to latent and sensible heat transport cannot be negligible in the entropy balance model. The zonally averaged two-latitude and ten-latitude models with ten radiative heating levels are used to find climates at minima of the entropy exchange rate. The models are examined under two different conditions of the water vapor distribution: one is the case with a given distribution of absolute humidity and the other the case with a given distribution of relative humidity. Multiple minima are found in the former case, while no minima in the latter case. In the former case, one of the minima corresponds to a climate with distributions of temperature and cloud amount similar to those in the present climate. However, it does not correspond to the least minimum of the entropy exchange rate. It is demonstrated that climates at minimum entropy exchange are very sensitive to the parameterization of the humidity distribution.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.