Despite deep neural networks have demonstrated extraordinary power in various applications, their superior performances are at expense of high storage and computational costs. Consequently, the acceleration and compression of neural networks have attracted much attention recently. Knowledge Transfer (KT), which aims at training a smaller student network by transferring knowledge from a larger teacher model, is one of the popular solutions. In this paper, we propose a novel knowledge transfer method by treating it as a distribution matching problem. Particularly, we match the distributions of neuron selectivity patterns between teacher and student networks. To achieve this goal, we devise a new KT loss function by minimizing the Maximum Mean Discrepancy (MMD) metric between these distributions. Combined with the original loss function, our method can significantly improve the performance of student networks. We validate the effectiveness of our method across several datasets, and further combine it with other KT methods to explore the best possible results. Last but not least, we fine-tune the model to other tasks such as object detection. The results are also encouraging, which confirm the transferability of the learned features.
The data suggest that low pollen viability, short pollen longevity and short pistil receptivity all appear to contribute to the low seed production typical of this important forage crop.
Quantum dots (QDs), as unique nanoparticle probes, have been used in in vivo fluorescence imaging such as cancers. Due to the novel characteristics in fluorescence, QDs represent a family of promising substances to be used in experimental and clinical imaging. Thus far, the toxicity and harmful health effects from exposure (including environmental exposure) to QDs are not recognized, but are largely concerned by the public. To assess the biological effects of QDs, we established a mouse model of acute and chronic exposure to QDs. Results from the present study suggested that QD particles could readily spread into various organs, and liver was the major organ for QD accumulation in mice from both the acute and chronic exposure. QDs caused significant impairments to livers from mice with both acute and chronic QD exposure as reflected by morphological alternation to the hepatic lobules and increased oxidative stress. Moreover, QDs remarkably induced the production of intracellular reactive oxygen species (ROS) along with cytotoxicity, as characterized by a significant increase of the malondialdehyde (MDA) level within hepatocytes. However, the increase of the MDA level in response to QD treatment could be partially blunted by the pre-treatment of cells with beta-mercaptoethanol (β-ME). These data suggested ROS played a crucial role in causing oxidative stress-associated cellular damage from QD exposure; nevertheless other unidentified mediators might also be involved in QD-mediated cellular impairments. Importantly, we demonstrated that the hepatoxicity caused by QDs in vivo and in vitro was much greater than that induced by cadmium ions at a similar or even a higher dose. Taken together, the mechanism underlying QD-mediated biological influences might derive from the toxicity of QD particles themselves, and from free cadmium ions liberated from QDs as well.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.