Mask-guided BERT for Few Shot Text Classification

Liao, Wenxiong; Liu, Zhengliang; Dai, Haixing; Wu, Zihao; Zhang, Yiyang; Huang, Xiaoke; Chen, Yuzhong; Jiang, Xi; Zhu, Dajiang; Liu, Tianming; Li, Sheng; Li, Xiang; Cai, Hao

doi:10.48550/arxiv.2302.10447

Cited by 3 publications

(3 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Datasets used to train language models can be substantial (1,29), often reaching hundreds of gigabytes, and they draw from various sources and domains (30)(31)(32)(33)(34). Consequently, even when trained on public data, these datasets can contain sensitive information, such as personally identifiable information (PII) including names, phone numbers, and addresses.…”

Section: The Emperor's New Clothes: Privacy Concernsmentioning

confidence: 99%

Surviving ChatGPT in healthcare

Liu,

Zhang,

et al. 2024

Front. Radiol.

View full text Add to dashboard Cite

At the dawn of of Artificial General Intelligence (AGI), the emergence of large language models such as ChatGPT show promise in revolutionizing healthcare by improving patient care, expanding medical access, and optimizing clinical processes. However, their integration into healthcare systems requires careful consideration of potential risks, such as inaccurate medical advice, patient privacy violations, the creation of falsified documents or images, overreliance on AGI in medical education, and the perpetuation of biases. It is crucial to implement proper oversight and regulation to address these risks, ensuring the safe and effective incorporation of AGI technologies into healthcare systems. By acknowledging and mitigating these challenges, AGI can be harnessed to enhance patient care, medical knowledge, and healthcare processes, ultimately benefiting society as a whole.

show abstract

Section: The Emperor's New Clothes: Privacy Concernsmentioning

confidence: 99%

Surviving ChatGPT in healthcare

Liu,

Zhang,

et al. 2024

Front. Radiol.

View full text Add to dashboard Cite

show abstract

“…Promptbased methods guide large language models to predict correct results by designing templates [8], [9], [10], [11]. Model design methods guide the model to learn from few-shot samples by changing the structure of the model [78]. Data augmentation uses similar characters [22], similar word semantics [30], [31], or knowledge base [54], [79] to expand samples.…”

Section: Few-shot Text Classificationmentioning

confidence: 99%

AugGPT: Leveraging ChatGPT for Text Data Augmentation

Dai¹,

Liu²,

Liao³

et al. 2023

Preprint

View full text Add to dashboard Cite

Text data augmentation is an effective strategy for overcoming the challenge of limited sample sizes in many natural language processing (NLP) tasks. This challenge is especially prominent in the few-shot learning scenario, where the data in the target domain is generally much scarcer and of lowered quality. A natural and widely-used strategy to mitigate such challenges is to perform data augmentation on the training data to better capture the data invariance and increase the sample size. However, current text data augmentation methods either can not ensure the correct labeling of the generated data (lacking faithfulness) or can not ensure sufficient diversity in the generated data (lacking completeness), or both. Inspired by the recent success of large language models, especially the development of ChatGPT, which demonstrated improved language comprehension abilities, in this work, we propose a text data augmentation approach based on ChatGPT (named ChatAug). ChatGPT is trained on data with unparalleled linguistic richness and employs a reinforcement training process with large-scale human feedback, which endows the model with affinity to the naturalness of human language. Our text data augmentation approach ChatAug rephrases each sentence in the training samples into multiple conceptually similar but semantically different samples. The augmented samples can then be used in downstream model training. Experiment results on few-shot learning text classification tasks show the superior performance of the proposed ChatAug approach over state-of-the-art text data augmentation methods in terms of testing accuracy and distribution of the augmented samples.

show abstract

“…Over time, the scale of these models has grown exponentially. Early examples of language models include BERT [2], T5 [3], GPT-1 [4], GPT-2 [5] and various BERT variants [6,7]. In addition, there exists a multitude of domain-specific BERT variants that are tailored to optimize performance in distinct fields of study or industry [8][9][10].…”

Section: Introductionmentioning

confidence: 99%

School Mathematics Textbooks in China

Wang¹

2015

East China Normal University Scientific Reports

View full text Add to dashboard Cite

Let µ and ν be two probability measures on R d , where µ(dx) = e −V (x) dx R d e −V (x) dx for some V ∈ C 1 (R d ). Explicit sufficient conditions on V and ν are presented such that µ * ν satisfies the log-Sobolev, Poincaré and super Poincaré inequalities. In particular, if V (x) = λ|x| 2 for some λ > 0 and ν(e λθ|•| 2 ) < ∞ for some θ > 1, then µ * ν satisfies the log-Sobolev inequality. This improves and extends the recent results on the log-Sobolev inequality derived in [20] for convolutions of the Gaussian measure and compactly supported probability measures. On the other hand, it is well known that the log-Sobolev inequality for µ * ν implies ν(e ε|•| 2 ) < ∞ for some ε > 0. RésuméDes conditions explicites suffisantes sur V et ν sont présentées telles que µ * ν satisfait des inégalités de Sobolev logarithmique, de Poincaré et de super-Poincaré. En particulier, si V (x) = λ|x| 2 pour quelque λ > 0 et ν(e λθ|•| 2 ) < ∞ avec θ > 1, alors µ * ν satisfait l'inégalité de Sobolev logarithmique. Cela améliore et étend des résultats récents sur l'inégalité de Sobolev logarithmique obtenus dans [20] pour des convolutions de la mesure de Gauss et des mesures de probabilité à support compact. D'autre part, il est bien connu que l'inégalité de Sobolev logarithmique pour µ * ν implique ν(e ε|•| 2 ) < ∞ pour quelque ε > 0.

show abstract

Mask-guided BERT for Few Shot Text Classification

Cited by 3 publications

References 44 publications

Surviving ChatGPT in healthcare

Surviving ChatGPT in healthcare

AugGPT: Leveraging ChatGPT for Text Data Augmentation

School Mathematics Textbooks in China

Contact Info

Product

Resources

About