Generative Pre-Trained Transformer (GPT) in Research: A Systematic Review on Data Augmentation

Abstract

GPT (Generative Pre-trained Transformer) represents advanced language models that have significantly reshaped the academic writing landscape. These sophisticated language models offer invaluable support throughout all phases of research work, facilitating idea generation, enhancing drafting processes, and overcoming challenges like writer’s block. Their capabilities extend beyond conventional applications, contributing to critical analysis, data augmentation, and research design, thereby elevating the efficiency and quality of scholarly endeavors. Strategically narrowing its focus, this review explores alternative dimensions of GPT and LLM applications, specifically data augmentation and the generation of synthetic data for research. Employing a meticulous examination of 412 scholarly works, it distills a selection of 77 contributions addressing three critical research questions: (1) GPT on Generating Research data, (2) GPT on Data Analysis, and (3) GPT on Research Design. The systematic literature review adeptly highlights the central focus on data augmentation, encapsulating 48 pertinent scholarly contributions, and extends to the proactive role of GPT in critical analysis of research data and shaping research design. Pioneering a comprehensive classification framework for “GPT’s use on Research Data”, the study classifies existing literature into six categories and 14 sub-categories, providing profound insights into the multifaceted applications of GPT in research data. This study meticulously compares 54 pieces of literature, evaluating research domains, methodologies, and advantages and disadvantages, providing scholars with profound insights crucial for the seamless integration of GPT across diverse phases of their scholarly pursuits.

Deep Learning: Historical Overview from Inception to Actualization, Models, Applications and Future Trends

“…Casey and Michal K. ( 2023) is a review study on GPT models. Also, (Sufi, 2024) reviewed the application of GPTs to research capturing data augmentation and synthetic data generation. Several efforts are ongoing to improve the performance of transformer models in terms of efficiency, robustness, interpretability and adaptability (OpenAI, 2018).…”

A Sustainable Way Forward: Systematic Review of Transformer Technology in Social-Media-Based Disaster Analytics

“…This meticulous review and analysis facilitated the development of a novel classification scheme, categorizing the aforementioned 53 publications into 10 distinct categories. Additionally, it is noteworthy that beyond the creation of this innovative classification scheme for existing literature concerning the utilization of transformer technology in social-media-based disaster analytics, this study presents a methodological and systematic review of the subject of employing contemporary AI-based tools such as Litmaps [16,17].…”

AGSAM: Agent-Guided Segment Anything Model for Automatic Segmentation in Few-Shot Scenarios

“…The recent success of Generative Pre-Trained Transformer (GPT) models in few-shot tasks [8][9][10], pre-trained on vast datasets, has inspired the development of the Segment Anything Model (SAM), a model trained extensively on data to encode and decode feature for segmentation, exhibiting remarkable few-shot and even zero-shot capabilities [11]. SAM-Med2D [12] bridges the gap between SAM's proficiency in natural images and its application in medical 2D image analysis.…”