Quantifying Sex Bias in Clinical Studies at Scale With Automated Data Extraction

Feldman, Sergey; Ammar, Waleed; Lo, Kyle; Trepman, Elly; Zuylen, Madeleine van; Etzioni, Oren

doi:10.1001/jamanetworkopen.2019.6700

Cited by 112 publications

(88 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been reported that different forms of bias exist in medical studies. Examples include that the more clinical studies involve males than females, the difference styles in which male and female patients report their pain and medical complaints, as well how male and female doctors record these complaints in medical reports (Feldman et al, 2019; Fillingim, King, Ribeiro‐Dasilva, Rahim‐Williams, & Riley, 2009). Suresh and Guttag (2019) categorized these types of bias as Representation bias and Aggregation Bias in their framework of bias.…”

Section: Discussionmentioning

confidence: 99%

Word embeddings for biomedical natural language processing: A survey

Chiu

Baker

2020

Language and Linguist. Compass

View full text Add to dashboard Cite

Word representations are mathematical objects that capture the semantic and syntactic properties of words in a way that is interpretable by machines. Recently, encoding word properties into low‐dimensional vector spaces using neural networks has become increasingly popular. Word embeddings are now used as the main input to natural language processing (NLP) applications, achieving cutting‐edge results. Nevertheless, most word‐embedding studies are carried out with general‐domain text and evaluation datasets, and their results do not necessarily apply to text from other domains (e.g., biomedicine) that are linguistically distinct from general English. To achieve maximum benefit when using word embeddings for biomedical NLP tasks, they need to be induced and evaluated using in‐domain resources. Thus, it is essential to create a detailed review of biomedical embeddings that can be used as a reference for researchers to train in‐domain models. In this paper, we review biomedical word embedding studies from three key aspects: the corpora, models and evaluation methods. We first describe the characteristics of various biomedical corpora, and then compare popular embedding models. After that, we discuss different evaluation methods for biomedical embeddings. For each aspect, we summarize the various challenges discussed in the literature. Finally, we conclude the paper by proposing future directions that will help advance research into biomedical embeddings.

show abstract

Section: Discussionmentioning

confidence: 99%

Word embeddings for biomedical natural language processing: A survey

Chiu

Baker

2020

Language and Linguist. Compass

View full text Add to dashboard Cite

show abstract

“…Recently published related work has used machine learning to extract sex data automatically from a large set of articles describing RCTs, finding evidence of systematic underrepresentation of women in trials. [32]Here, we broaden this approach by applying machine learning to infer the conditions and outcomes under study in trials and extracting corresponding sample sizes. By using our previously validated model [12] to automatically recognize reports of RCTs, we are able to continuously surveil the literature to maintain an up-to-date, comprehensive view of the evidence, which we make publicly available to facilitate further research.…”

Section: Discussionmentioning

confidence: 99%

State of the evidence: a survey of global disparities in clinical trials

Marshall

L’Esperence

Marshall

et al. 2020

Preprint

View full text Add to dashboard Cite

Introduction Ideally, health conditions causing the greatest global disease burden should attract increased research attention. We conducted a comprehensive global study investigating the number of randomised controlled trials (RCTs) published on different health conditions, and how this compares with the global disease burden that they impose. Methods We use machine learning to monitor PubMed daily, and find and analyse RCT reports. We assessed RCTs investigating the leading causes of morbidity and mortality from the Global Burden of Disease study. Using regression models, we compared numbers of actual RCTs in different health conditions to numbers predicted from their global disease burden (Disability-Adjusted Life Years [DALYs]). We investigated whether RCT numbers differed for conditions disproportionately affecting countries with lower socio-economic development. Results We estimate 463,000 articles describing RCTs (95% prediction interval 439,000-485,000) were published from 1990 to July 2020. RCTs recruited a median of 72 participants (interquartile range 32-195). 82% of RCTs were conducted by researchers in the top fifth of countries by socio-economic development. As DALYs increased for a particular health condition by 10%, the number of RCTs in the same year increased by 5% (3.2%-6.9%), but the association was weak (adjusted R2=0.13). Conditions disproportionately affecting countries with lower socio-economic development, including respiratory infections and tuberculosis (7 thousand RCTs below predicted) and enteric infections (10 thousand RCTs below predicted), appear relatively under-researched for their disease burden. Each 10% shift in DALYs towards countries with low and middle socio-economic development was associated with a 4% reduction in RCTs (3.7%-4.9%). These disparities have not changed substantially over time. Conclusion Research priorities are not well optimized to reduce the global burden of disease. Most RCTs are produced by highly developed countries, and the health needs of these countries have been, on average, favoured.

show abstract

“…In 1993, the policies excluding women were revoked and the National Institutes of Health (NIH) Revitalization Act was passed to increase inclusion of women and minorities in clinical research. This has improved inclusion of women, but clinical studies continue to show sex bias against female participants [ 2 – 4 ]. Additionally, preclinical studies are critical to the drug development process [ 5 ]; however, there is limited reporting of sex in both rodent [ 6 , 7 ] and cell line research [ 8 ].…”

Section: Introductionmentioning

confidence: 99%

Large-scale labeling and assessment of sex bias in publicly available expression data

2021

View full text Add to dashboard Cite

Background Women are at more than 1.5-fold higher risk for clinically relevant adverse drug events. While this higher prevalence is partially due to gender-related effects, biological sex differences likely also impact drug response. Publicly available gene expression databases provide a unique opportunity for examining drug response at a cellular level. However, missingness and heterogeneity of metadata prevent large-scale identification of drug exposure studies and limit assessments of sex bias. To address this, we trained organism-specific models to infer sample sex from gene expression data, and used entity normalization to map metadata cell line and drug mentions to existing ontologies. Using this method, we inferred sex labels for 450,371 human and 245,107 mouse microarray and RNA-seq samples from refine.bio. Results Overall, we find slight female bias (52.1%) in human samples and (62.5%) male bias in mouse samples; this corresponds to a majority of mixed sex studies in humans and single sex studies in mice, split between female-only and male-only (25.8% vs. 18.9% in human and 21.6% vs. 31.1% in mouse, respectively). In drug studies, we find limited evidence for sex-sampling bias overall; however, specific categories of drugs, including human cancer and mouse nervous system drugs, are enriched in female-only and male-only studies, respectively. We leverage our expression-based sex labels to further examine the complexity of cell line sex and assess the frequency of metadata sex label misannotations (2–5%). Conclusions Our results demonstrate limited overall sex bias, while highlighting high bias in specific subfields and underscoring the importance of including sex labels to better understand the underlying biology. We make our inferred and normalized labels, along with flags for misannotated samples, publicly available to catalyze the routine use of sex as a study variable in future analyses.

show abstract

Quantifying Sex Bias in Clinical Studies at Scale With Automated Data Extraction

Cited by 112 publications

References 36 publications

Word embeddings for biomedical natural language processing: A survey

Word embeddings for biomedical natural language processing: A survey

State of the evidence: a survey of global disparities in clinical trials

Large-scale labeling and assessment of sex bias in publicly available expression data

Contact Info

Product

Resources

About