Demographic Representation in 3 Leading Artificial Intelligence Text-to-Image Generators

Abstract

ImportanceThe progression of artificial intelligence (AI) text-to-image generators raises concerns of perpetuating societal biases, including profession-based stereotypes.ObjectiveTo gauge the demographic accuracy of surgeon representation by 3 prominent AI text-to-image models compared to real-world attending surgeons and trainees.Design, Setting, and ParticipantsThe study used a cross-sectional design, assessing the latest release of 3 leading publicly available AI text-to-image generators. Seven independent reviewers categorized AI-produced images. A total of 2400 images were analyzed, generated across 8 surgical specialties within each model. An additional 1200 images were evaluated based on geographic prompts for 3 countries. The study was conducted in May 2023. The 3 AI text-to-image generators were chosen due to their popularity at the time of this study. The measure of demographic characteristics was provided by the Association of American Medical Colleges subspecialty report, which references the American Medical Association master file for physician demographic characteristics across 50 states. Given changing demographic characteristics in trainees compared to attending surgeons, the decision was made to look into both groups separately. Race (non-White, defined as any race other than non-Hispanic White, and White) and gender (female and male) were assessed to evaluate known societal biases.ExposuresImages were generated using a prompt template, “a photo of the face of a [blank]”, with the blank replaced by a surgical specialty. Geographic-based prompting was evaluated by specifying the most populous countries on 3 continents (the US, Nigeria, and China).Main Outcomes and MeasuresThe study compared representation of female and non-White surgeons in each model with real demographic data using χ2, Fisher exact, and proportion tests.ResultsThere was a significantly higher mean representation of female (35.8% vs 14.7%; P < .001) and non-White (37.4% vs 22.8%; P < .001) surgeons among trainees than attending surgeons. DALL-E 2 reflected attending surgeons’ true demographic data for female surgeons (15.9% vs 14.7%; P = .39) and non-White surgeons (22.6% vs 22.8%; P = .92) but underestimated trainees’ representation for both female (15.9% vs 35.8%; P < .001) and non-White (22.6% vs 37.4%; P < .001) surgeons. In contrast, Midjourney and Stable Diffusion had significantly lower representation of images of female (0% and 1.8%, respectively; P < .001) and non-White (0.5% and 0.6%, respectively; P < .001) surgeons than DALL-E 2 or true demographic data. Geographic-based prompting increased non-White surgeon representation but did not alter female representation for all models in prompts specifying Nigeria and China.Conclusion and RelevanceIn this study, 2 leading publicly available text-to-image generators amplified societal biases, depicting over 98% surgeons as White and male. While 1 of the models depicted comparable demographic characteristics to real attending surgeons, all 3 models underestimated trainee representation. The study suggests the need for guardrails and robust feedback systems to minimize AI text-to-image generators magnifying stereotypes in professions such as surgery.

Demographic Inaccuracies and Biases in the Depiction of Patients by Artificial Intelligence Text-to-Image Generators

“…Female individuals as well as Asian, BAA, HL, NHPI, and AIAN individuals were depicted as being more overweight compared to male and White individuals, respectively. Such inaccuracies raise concern about the role of AI in amplifying misconceptions in healthcare 14 given its large numbers of users and use cases [3][4][5][6][7][8][9] .…”

“…In fact, there was no over-representation of any sex in the images of the two text-to-image generators. This may be interpreted as a positive sign as sex/gender biases have been a common phenomenon in generative AI algorithms 12,14 . On the other hand, achieving accurate demographic representation by applying bias mitigation strategies seems challenging and representative training data may be necessary.…”

“…Yet, recent research has shown that also AI algorithms are susceptible to biases based on sex and gender as well as race and ethnicity [12][13][14] . Especially females and people of color were shown to be under-represented in established training datasets 12 .…”

“…Especially females and people of color were shown to be under-represented in established training datasets 12 . In consequence AI algorithms tend to misclassify and misinterpret such images [12][13][14] . In addition, some diseases such as infectious 15,16 , psychiatric 17,18 , or internal medicine diseases 19 carry disease-related stigmas with profound effects on patients.…”

Can artificial intelligence help for scientific illustration? Details matter

“…However, being trained on the current distribution of available imagery, generative AI tools pose a risk to perpetuate and amplify existing biases and stereotypes of oversimplified social categorization. For example, when prompted to generate images of surgeons, over 98% are depicted as White and male [ 12 ]. Contrarily, when instructed to create an image of suffering children, the patients are unvaryingly depicted as Black [ 13 ].…”