Foundation models for generalist medical artificial intelligence

Abstract

The exceptionally rapid development of highly flexible, reusable artificial intelligence (AI) models is likely to usher in newfound capabilities in medicine. We propose a new paradigm for medical AI, which we refer to as generalist medical AI (GMAI). GMAI models will be capable of carrying out a diverse set of tasks using very little or no task-specific labelled data. Built through self-supervision on large, diverse datasets, GMAI will flexibly interpret different combinations of medical modalities, including data from imaging, electronic health records, laboratory results, genomics, graphs or medical text. Models will in turn produce expressive outputs such as free-text explanations, spoken recommendations or image annotations that demonstrate advanced medical reasoning abilities. Here we identify a set of high-impact potential applications for GMAI and lay out specific technical capabilities and training datasets necessary to enable them. We expect that GMAI-enabled applications will challenge current strategies for regulating and validating AI devices for medicine and will shift practices associated with the collection of large medical datasets.

Artificial intelligence and the impact on medical genetics

“…Newer foundational AI models will be able to ingest, analyze, and generate diverse data types. As intuitive user interfaces are layered on top of these models, the geneticist of the future will operate in ways that are very different from today (Lee et al, 2023;Moor et al, 2023). In this piece, we briefly outline our predictions pertaining to the field of medical genetics.…”

The Challenges for Regulating Medical Use of ChatGPT and Other Large Language Models

“…What has changed is the public availability and interest in large language models (LLMs), most notably OpenAI’s ChatGPT and Google’s Bard, with potential applications in medicine. These LLMs are large (1 trillion parameters for GPT-4 and growing), general rather than trained for a specific task (though fine-tuning is possible), autoregressive (using only past data to predict future data), and foundational (likely to be the seed for many further devices and applications) . There has also been interest in creating LLMs with more curated medical training data, such as Google’s Med-PaLM 2.…”

Retrieve, Summarize, and Verify: How Will ChatGPT Affect Information Seeking from the Medical Literature?

“…Recent work has discussed applications of ChatGPT for medical education and clinical decision support. [2][3][4] However, health care professionals should be aware of the drawbacks and limitations-and potential capabilities -of using ChatGPT and similar LLMs to interact with medical knowledge.…”