Although distributed learning has increasingly gained attention in terms of effectively utilizing local devices for data privacy enhancement, recent studies show that publicly shared gradients in the training process can reveal the private training data (gradient leakage) to a third party. However, so far there hasn't been any systematic study of the gradient leakage mechanism of the Transformer based language models. In this paper, as the first attempt, we formulate the gradient attack problem on the Transformer-based language models and propose a gradient attack algorithm, TAG, to recover the local training data. Experimental results on Transformer, TinyBERT 4 , TinyBERT 6 , BERT BASE , and BERT LARGE using GLUE benchmark show that compared with DLG (Zhu et al., 2019), TAG works well on more weight distributions in recovering private training data and achieves 1.5× Recover Rate and 2.5× ROUGE-2 over prior methods without the need of ground truth label. TAG can obtain up to 88.9% tokens and up to 0.93 cosine similarity in token embeddings from private training data by attacking gradients on CoLA dataset. In addition, TAG is stronger than previous approaches on larger models, smaller dictionary size, and smaller input length.
Flexible pressure sensors have attracted a great deal of attention due to their significant potential for applications in electronic skins, artificial intelligence and wearable health care devices. It is still challenging to obtain the flexible pressure sensor with high sensitivity and large linear measuring range in a low cost and facile way. In this paper, the composite dielectrics ink based on thermal expansion microcapsules (TEMs), silver nanowires (Ag NWs) and polydimethylsiloxane was employed to improve the performance of the flexible capacitive pressure sensor. The screen printing method was used to prepare the electrodes and microstructural composite dielectric layer. The results indicated that the flexible sensor with composite dielectrics of 1 wt.% TEMs and 0.5 wt.% Ag NWs demonstrated the excellent performance including the maximum sensitivity of 2.1 kPa−1 and wide linear pressure range. The dramatic improvement in the sensor’s sensitivity and linear pressure range could be attributed to the synergetic effects of the TEMs controllable microstructure and relative permittivity increase of composite dielectrics under pressure. In addition, the full printed flexible pressure sensor showed its limit of detection of 1.3 Pa, responding time of 50 ms, proximity sensing distance of 24 cm and good mechanical durability over 3600 cyclic compress–release testing. To our best knowledge, these characteristics are superior to the printed capacitive flexible sensor in reporting. In this paper, the full printed flexible pressure sensor demonstrates it is a good candidate to be applied in the field of E-skin, pressure mapping and wearable health care devices, etc.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.