Binarized Neural Networks (BNNs) can significantly reduce the inference latency and energy consumption in resource-constrained devices due to their pure-logical computation and fewer memory accesses. However, training BNNs is difficult since the activation flow encounters degeneration, saturation, and gradient mismatch problems. Prior work alleviates these issues by increasing activation bits and adding floating-point scaling factors, thereby sacrificing BNN's energy efficiency. In this paper, we propose to use distribution loss to explicitly regularize the activation flow, and develop a framework to systematically formulate the loss. Our experiments show that the distribution loss can consistently improve the accuracy of BNNs without losing their energy benefits. Moreover, equipped with the proposed regularization, BNN training is shown to be robust to the selection of hyper-parameters including optimizer and learning rate.
Can we automatically design a Convolutional Network (Con-vNet) with the highest image classification accuracy under the latency constraint of a mobile device? Neural architecture search (NAS) has revolutionized the design of hardware-efficient ConvNets by automating this process. However, the NAS problem remains challenging due to the combinatorially large design space, causing a significant searching time (at least 200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a novel differentiable NAS method for designing hardware-efficient ConvNets in less than 4 hours. Our contributions are as follows: 1. Single-path search space: Compared to previous differentiable NAS methods, Single-Path NAS uses one singlepath over-parameterized ConvNet to encode all architectural decisions with shared convolutional kernel parameters, hence drastically decreasing the number of trainable parameters and the search cost down to few epochs. 2. Hardware-efficient ImageNet classification: Single-Path NAS achieves 74.96% top-1 accuracy on ImageNet with 79ms latency on a Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with similar inference latency constraints (≤ 80ms). 3. NAS efficiency: Single-Path NAS search cost is only 8 epochs (30 TPUhours), which is up to 5,000× faster compared to prior work. 4. Reproducibility: Unlike all recent mobile-efficient NAS methods which only release pretrained models, we open-source our entire codebase at: https://github.com/dstamoulis/single-path-nas.
Palladium (Pd) nanocrystals have been synthesized by using formic acid as the reducing agent at room temperature. When the concentration of formic acid was increased continuously, the size of Pd nanocrystals first decreased to a minimum and then increased slightly again. The products have been investigated by a series of techniques, including X-ray diffraction, high-resolution transmission electron microscopy (HRTEM), UV-vis absorption, and electrochemical measurements. The formation of Pd nanocrystals is proposed to be closely related to the dynamical imbalance of the growth and dissolution rate of Pd nanocrystals associated with the adsorption of formate ions onto the surface of the intermediates. It is found that small Pd nanocrystals showed blue-shifted adsorption peaks compared with large ones. Pd nanocrystals with the smallest size display the highest electrocatalytic activity for the electrooxidation of formic acid and ethanol on the basis of cyclic voltammetry and chronoamperometric data. It is suggested that both the electrochemical active surface area and the small size effect are the key roles in determining the electrocatalytic performances of Pd nanocrystals. A "dissolution-deposition-aggregation" process is proposed to explain the variation of the electrocatalytic activity during the electrocatalysis according to the HRTEM characterization.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.