Transformer-based pre-trained language models like BERT, though powerful in many tasks, are expensive in both memory and computation, due to their large number of parameters. Previous works show that some parameters in these models can be pruned away without severe accuracy drop. However, these redundant features contribute to a comprehensive understanding of the training data and removing them weakens the model's representation ability. In this paper, we propose GhostBERT, which generates more features with very cheap operations from the remaining features. In this way, GhostBERT has similar memory and computational cost as the pruned model, but enjoys much larger representation power. The proposed ghost module can also be applied to unpruned BERT models to enhance their performance with negligible additional parameters and computation. Empirical results on the GLUE benchmark on three backbone models (i.e., BERT, RoBERTa and ELECTRA) verify the efficacy of our proposed method.
The wavelength variation of a laser-dye-type random laser is observed experimentally. It is found that the emitting wavelength of a random laser changes with the change of concentration of the gain material. Also, the actual radiation wavelength is influenced by the pumping rate of the source, the cavity competition and the concentration of scatterers.
The two main fixing methods in the assembly process of optical components are adhesive bonding and thread fastening. For both methods, boundary loading effect is generated on optical component, which results in stress distribution in it. The quantitative prediction and control of the stress distribution in optical component is not available because the boundary load induced by adhesive or thread is difficult to control and test quantitatively. Up to now, the interrelationship and variation between the boundary load and the internal stress in optical component have not been effectively studied. Based on the above requirements, quantitative radial boundary loading and real-time measuring methods for optical component are proposed in this paper. Quantitative interrelationship between the boundary load and the stress distribution and surface form in optical component can be obtained by integrating the stress and surface measurement devices with radial boundary loading system. This innovative loading and measuring device is suitable for various sizes of optical components, and it solves the quantitative application and control problems during loading, measuring, and real-time detection of adhesive curing force.
Thickness-shear vibrations of an AT-cut quartz resonator due to a liquid layer are studied. Closed form solutions are obtained with electrodes considered. Effect of the shear stiffness of electrodes on frequency shifts is presented.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.