We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models, which achieve the state-of-the-art privacy versus utility tradeoffs on many standard NLP tasks. We propose a meta-framework for this problem, inspired by the recent success of highly parameter-efficient methods for fine-tuning. Our experiments show that differentially private adaptations of these approaches outperform previous private algorithms in three important dimensions: utility, privacy, and the computational and memory cost of private training. On many commonly studied datasets, the utility of private models approaches that of non-private models. For example, on the MNLI dataset we achieve an accuracy of 87.8% using RoBERTa-Large and 83.5% using RoBERTa-Base with a privacy budget of ε = 6.7. In comparison, absent privacy constraints, RoBERTa-Large achieves an accuracy of 90.2%. Our findings are similar for natural language generation tasks. Privately fine-tuning with DART, GPT-2-Small, GPT-2-Medium, GPT-2-Large, and GPT-2-XL achieve BLEU scores of 38.5, 42.0, 43.1, and 43.8 respectively (privacy budget of ε = 6.8, δ = 1e-5) whereas the non-private baseline is 48.1. All our experiments suggest that larger models are better suited for private fine-tuning: while they are well known to achieve superior accuracy non-privately, we find that they also better maintain their accuracy when privacy is introduced.

No abstract

No abstract

In unconventional water-wet gas reservoirs with very low permeability, water entrapment or blockage can occur near the wellbore due to the capillary end effect, resulting in low gas production. A reduction in capillary forces through wettability alteration of reservoir rock surface is proposed as an effective approach to reduce water blockage and enhance gas production. The method can be applied to accelerating dewatering and preventing drilling and fracturing fluid leak-off as well. Analytical models for steady-state water-gas linear and radial flows are developed in the current paper. The effects of contact angle on capillary pressure and relative permeabilities have been included. The new model is validated using experimental data. Applications to fully and partially treated regimes show the competition between viscous and capillary effects on productivity of gas and water, which leads to an optimal contact angle for the maximum productivity index for each phase. This study shows the potential for optimising unconventional gas productivity through wettability control. Application of nanotechnology to rock wettability alteration is proposed.

This paper presents a low power continuous time 2 nd order Low Pass Butterworth filter operating at power supply of 0.5V suitably designed for biomedical applications. A 3-dB bandwidth of 100 Hz using technology node of 0.18μm is achieved. The operational transconductance amplifier is a significant building block in continuous time filter design. To achieve necessary voltage headroom a pseudo-differential architecture is used to design bulk driven transconductor. In contrast, to the gate-driven OTA bulk-driven have the ability to operate over a wide input range. The output common mode voltage of the transconductor is set by a Common Mode Feedback (CMFB) circuit. The simulation results show that the filter has a peak-to-peak signal swing of 150mV (differential) for 1% THD, a dynamic range of 74.62 dB and consumes a total power of 0.225μW when operating at a supply voltage of 0.5V. The Figure of Merit (FOM) achieved by the filter is 0.055 fJ, lowest among similar low-voltage filters found in the literature.