Zero-Shot Action Recognition with ChatGPT-Based Instruction

Wu, Nan; Kera, Hiroshi; Kawamoto, Kazuhiko

doi:10.1007/978-981-99-7593-8_3

Cited by 3 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These robust performance metrics serve as a validation of the efficacy of our pipeline in the accurate extraction of detailed information from multiple myeloma clinical trial studies. Additionally, this study systematically explored the zero-shot capabilities of GPT-4 [20,32], particularly in tasks such as interventions and the recognition of treatment outcome entities. These tasks included the association of outcomes with corresponding values in each cohort and the classification of cohorts, all seamlessly executed through the use of efficiently designed single-turn prompts.…”

Section: Discussionmentioning

confidence: 99%

“…Within the realm of information extraction from clinical documents, various LLMs have been deployed in diverse applications, such as analyzing radiology reports [17–19] and electronic health records [15]. In the clinical trial domain, LLMs play crucial roles in trial information retrieval [20], criteria text generation [21], and clinical trial eligibility criteria analysis [22]. Despite recognized limitations, ChatGPT has also demonstrated value in supporting systematic literature reviews [23].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SEETrials: Leveraging Large Language Models for Safety and Efficacy Extraction in Oncology Clinical Trials

Lee,

Paek,

Huang

et al. 2024

Preprint

View full text Add to dashboard Cite

Objective Conference abstracts provide preliminary evidence for clinical trial outcomes. This study aims to develop an automatic extraction system to precisely extract and convert granular safety and efficacy information from abstracts into a computable format for timely downstream analysis. Materials and Methods We collected multiple myeloma clinical trial abstracts from ASCO, ASH, and PubMed (2012-2023) to develop SEETrials, a GPT-4 based system. Qualitative and quantitative evaluations were conducted. Descriptive data analysis on efficacy and safety entities was performed. The generalizability of the system was tested in three other cancer trial studies. Results SEETrials achieved precision, recall, and F1 of 95.5%, 93.9%, and 94.7% across 70 data elements. Analysis of 245 multiple myeloma clinical trial abstracts revealed variations in safety and efficacy entity distribution across different modalities and phases. Application to other cancer trial studies demonstrated consistent performance with precision, recall, and F1 of 96.9%, 95.4%, and 96.1%, respectively. Discussion Qualitative error analysis identified a common source of inaccuracy, notably in cohort identification and categorization. Phase1/2 studies prioritize early indications of treatment efficacy, while phase 2/3 studies emphasize prolonged effects. Adverse event distribution is consistent across phases, with noticeable increases in fatal events in later phases, suggesting longer follow-up or more patients may reveal severe adverse events missed in phase 1 safety studies. Conclusion SEETrials displayed high accuracy and generalizability to diverse drug modalities and disease domains. Its capacity to streamline large-scale dataset analysis is crucial for advancing clinical trial research, ensuring timely and accurate data extraction, and facilitating efficient dissemination.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

SEETrials: Leveraging Large Language Models for Safety and Efficacy Extraction in Oncology Clinical Trials

Lee,

Paek,

Huang

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, generative pretrained transformers such as Llama-2 21 and GPT-4 22 have been trained and applied to clinical NLP tasks in zero-shot or few-shot settings, which brings new inspirations to our system design. 23,24 Integrating various NLP tools into a unified pipeline poses significant challenges, notably managing component dependencies and system stability. Errors in early stages can affect downstream performance, evident from declines seen from named entity recognition to relation normalization.…”

Section: Discussionmentioning

confidence: 99%

Extracting Systemic Anticancer Therapy and Response Information From Clinical Notes Following the RECIST Definition

Zuo,

Kumar,

Shen

et al. 2024

JCO Clin Cancer Inform

View full text Add to dashboard Cite

PURPOSE The RECIST guidelines provide a standardized approach for evaluating the response of cancer to treatment, allowing for consistent comparison of treatment efficacy across different therapies and patients. However, collecting such information from electronic health records manually can be extremely labor-intensive and time-consuming because of the complexity and volume of clinical notes. The aim of this study is to apply natural language processing (NLP) techniques to automate this process, minimizing manual data collection efforts, and improving the consistency and reliability of the results. METHODS We proposed a complex, hybrid NLP system that automates the process of extracting, linking, and summarizing anticancer therapy and associated RECIST-like responses from narrative clinical text. The system consists of multiple machine learning–/deep learning–based and rule-based modules for diverse NLP tasks such as named entity recognition, assertion classification, relation extraction, and text normalization, to address different challenges associated with anticancer therapy and response information extraction. We then evaluated the system performances on two independent test sets from different institutions to demonstrate its effectiveness and generalizability. RESULTS The system used domain-specific language models, BioBERT and BioClinicalBERT, for high-performance therapy mentions identification and RECIST responses extraction and categorization. The best-performing model achieved a 0.66 score in linking therapy and RECIST response mentions, with end-to-end performance peaking at 0.74 after relation normalization, indicating substantial efficacy with room for improvement. CONCLUSION We developed, implemented, and tested an information extraction system from clinical notes for cancer treatment and efficacy assessment information. We expect this system will support future cancer research, particularly oncologic studies that focus on efficiently assessing the effectiveness and reliability of cancer therapeutics.

show abstract

“…In particular, Ope-nAI's fourth-generation Generative Pre-trained Transformer (GPT-4) has seen major improvements over previous models across a variety of benchmarks (OpenAI, 2023a). GPT-based models have already been investigated for a wide range of text mining tasks for research, including clinical (Hu et al, 2023), medical (Chen et al, 2023;Fink et al, 2023) and agricultural (Zhao et al, 2023), however, quantitative investigation into their potential in ecology has so far been lacking.…”

Section: Introductionmentioning

confidence: 99%

Large language models help facilitate the automated synthesis of information on potential pest controllers

Scheepens,

Millard,

Farrell

et al. 2024

Preprint

View full text Add to dashboard Cite

The body of ecological literature, which informs much of our knowledge of the global loss of biodiversity, has been experiencing rapid growth in recent decades. The increasing difficulty to synthesise this literature manually has simultaneously resulted in a growing demand for automated text mining methods. Within the domain of deep learning, large language models (LLMs) have been the subject of considerable attention in recent years by virtue of great leaps in progress and a wide range of potential applications, however, quantitative investigation into their potential in ecology has so far been lacking. In this work, we analyse the ability of GPT-4 to extract information about invertebrate pests and pest controllers from abstracts of a body of literature on biological pest control, using a bespoke, zero-shot prompt. Our results show that the performance of GPT-4 is highly competitive with other state-of-the-art tools used for taxonomic named entity recognition and geographic location extraction tasks. On a held-out test set, we show that species and geographic locations are extracted with F1-scores of 99.8% and 95.3%, respectively, and highlight that the model is able to distinguish very effectively between the primary roles of interest (predators, parasitoids and pests). Moreover, we demonstrate the ability of the model to effectively extract and predict taxonomic information across various taxonomic ranks, and to automatically correct spelling mistakes. However, we do report a small number of cases of fabricated information (hallucinations). As a result of the current lack of specialised, pre-trained ecological language models, general-purpose LLMs may provide a promising way forward in ecology. Combined with tailored prompt engineering, such models can be employed for a wide range of text mining tasks in ecology, with the potential to greatly reduce time spent on manual screening and labelling of the literature.

show abstract

Zero-Shot Action Recognition with ChatGPT-Based Instruction

Cited by 3 publications

References 15 publications

SEETrials: Leveraging Large Language Models for Safety and Efficacy Extraction in Oncology Clinical Trials

SEETrials: Leveraging Large Language Models for Safety and Efficacy Extraction in Oncology Clinical Trials

Extracting Systemic Anticancer Therapy and Response Information From Clinical Notes Following the RECIST Definition

Large language models help facilitate the automated synthesis of information on potential pest controllers

Contact Info

Product

Resources

About