Learning Boolean specifications

Bubeck, Uwe; Büning, Hans Kleine

doi:10.1016/j.artint.2015.09.003

Cited by 16 publications

(27 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is important to set up a search query that generates high precision at first to generate an initial document pool and taxonomy for bootstrapping. It is recommended to test various search algorithms using syntactic operators [32] by using, for example, Boolean [78], proximity and/or truncation operators (please note that retrieval algorithm syntax needs to be modified if you switch from one portal to another; NEAR/3 at ''Web of Science'' is equivalent to W/3 at ''ScienceDirect''). The query string from above, when used in various popular literature database portals (e.g., AIS eLibrary, IEEE Xplore, ScienceDirect, Web of Science, Wiley Online Library and SpringerLink), revealed 354 unique articles with high precision.…”

Section: Application and Evaluation Of The Stirl Methodology A Qmentioning

confidence: 99%

The Application of Artificial Intelligence Technologies as a Substitute for Reading and to Support and Enhance the Authoring of Scientific Review Articles

et al. 2019

View full text Add to dashboard Cite

To gain a comprehensive overview of new scientific findings with the enormous, ever-increasing amount of published information, we apply a new combinatorial approach that complements the process of reading scientific articles by supplementing artificial intelligence technologies. We present a combinatorial approach, which we illustrate in the form of a ''double funnel of artificial intelligence.'' Our approach suggests to largely increase the amount of data at the beginning of the data collection process and to subsequently clean and enrich the data set in order to gain much more knowledge at the end of the procedure compared to a ''classical'' literature review. We use natural language processing and text visualization techniques to uncover findings that are generally unbeknown to the human reader due to the inability to process very large amounts of text. By illustrating the individual steps using practical examples taken from use cases, we demonstrate the merits of our approach. With our methodology, we are able to reproduce findings from ''regular'' review papers; however, we discover additional and new findings in different fields, such as data science or medicine. We also point out the limitations of our approach. Finally, we make suggestions as to how the methodology could be further developed. INDEX TERMS Computational and artificial intelligence, document handling, fuzzy control, knowledge acquisition, pattern analysis, scientific publishing, text mining, text processing.

show abstract

Section: Application and Evaluation Of The Stirl Methodology A Qmentioning

confidence: 99%

The Application of Artificial Intelligence Technologies as a Substitute for Reading and to Support and Enhance the Authoring of Scientific Review Articles

et al. 2019

View full text Add to dashboard Cite

show abstract

“…New advances in generative artificial intelligence, specifically in Large Language Models (LLMs), could present a potential solution to this problem 5–9 . These AI models have the ability to engage in nuanced and complex conversations 10–12 , making them ideal candidates for extracting comprehensive patient histories and assisting physicians in generating differential diagnoses.…”

Section: Introductionmentioning

confidence: 99%

Guidelines For Rigorous Evaluation of Clinical LLMs For Conversational Reasoning

Johri,

Jeong,

Tran

et al. 2023

Preprint

View full text Add to dashboard Cite

Large Language Models (LLMs) show promise for medical diagnosis, but traditional evaluations using static exam questions overlook the complexity of real-world clinical dialogues. We introduce a multi-agent conversational framework where doctor-AI and patient-AI agents interact to diagnose medical conditions, evaluated by a grader-AI agent and medical experts. We assessed the diagnostic accuracy of GPT-4 and GPT-3.5, in conversational versus static settings using 140 cases focusing on skin diseases. Our study revealed a decline in diagnostic accuracy, unmasking key limitations in LLMs’ ability to integrate details from conversational interactions to improve diagnostic performance. We introduced Conversational Summarization, a technique that enhanced performance, and expert review identified deficiencies compared to human dermatologists in comprehensive history gathering, appropriate use of terminology, and reliability. Our findings advocate for nuanced, rigorous evaluation of LLMs before clinical integration, and our framework represents a significant advancement toward responsible testing methodologies in medicine.

show abstract

“…Contemporaneous DL models for natural language processing (NLP) are large language models (LLMs) such as generative pre‐trained transformer 4 (GPT‐4). These LLMs exhibit remarkable capabilities in understanding and processing of human‐written texts and can reason over a wide range of applications [5]. In radiology, a recent proof‐of‐concept study has demonstrated that this can be used to extract structured and quantitative information from unstructured reports written in plain language [6].…”

Section: Introductionmentioning

confidence: 99%

“…However, these studies developed their own standalone NLP system which was specifically tailored to the application (and potentially the dataset) at hand; also, they required training on large domain‐specific datasets. In contrast, modern LLMs such as GPT‐4 have zero‐shot capabilities [5], which means that they can be applied to complex tasks without any further training. This could allow users to extract structured data from pathology reports without making any changes to the LLM model, without any re‐training or fine‐tuning.…”

Section: Introductionmentioning

confidence: 99%

Extracting structured information from unstructured histopathology reports using generative pre‐trained transformer 4 (GPT‐4)

Truhn,

Loeffler,

Müller‐Franzes

et al. 2023

The Journal of Pathology

View full text Add to dashboard Cite

Deep learning applied to whole‐slide histopathology images (WSIs) has the potential to enhance precision oncology and alleviate the workload of experts. However, developing these models necessitates large amounts of data with ground truth labels, which can be both time‐consuming and expensive to obtain. Pathology reports are typically unstructured or poorly structured texts, and efforts to implement structured reporting templates have been unsuccessful, as these efforts lead to perceived extra workload. In this study, we hypothesised that large language models (LLMs), such as the generative pre‐trained transformer 4 (GPT‐4), can extract structured data from unstructured plain language reports using a zero‐shot approach without requiring any re‐training. We tested this hypothesis by utilising GPT‐4 to extract information from histopathological reports, focusing on two extensive sets of pathology reports for colorectal cancer and glioblastoma. We found a high concordance between LLM‐generated structured data and human‐generated structured data. Consequently, LLMs could potentially be employed routinely to extract ground truth data for machine learning from unstructured pathology reports in the future. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

show abstract

Learning Boolean specifications

Cited by 16 publications

References 12 publications

The Application of Artificial Intelligence Technologies as a Substitute for Reading and to Support and Enhance the Authoring of Scientific Review Articles

The Application of Artificial Intelligence Technologies as a Substitute for Reading and to Support and Enhance the Authoring of Scientific Review Articles

Guidelines For Rigorous Evaluation of Clinical LLMs For Conversational Reasoning

Extracting structured information from unstructured histopathology reports using generative pre‐trained transformer 4 (GPT‐4)

Contact Info

Product

Resources

About