Artificial intelligence and algorithmic bias: implications for health systems

Panch, Trishan; Mattie, Heather; Atun, Rifat

doi:10.7189/jogh.09.020318

Cited by 247 publications

(176 citation statements)

References 7 publications

Supporting

Mentioning

139

Contrasting

Unclassified

Order By: Relevance

“…Algorithms and artificial intelligence are at risk of exacerbating health inequalities by systematic misclassification. 40 We found that in our population, the proportion classified as at high risk according to national criteria was similar for those aged over 95 years as in those aged 65-69 years. Given that the proportion at high risk increases monotonically until the age of 85 years, there is a concern that this may reflect systematic underclassification of risk in the oldest old of our population, especially because the proportion at low risk paradoxically increases from the age of 85 years and onward.…”

Section: Discussionsupporting

confidence: 53%

Population Health Management to identify and characterise ongoing health need for high-risk individuals shielded from COVID-19: a cross-sectional cohort study

et al. 2020

View full text Add to dashboard Cite

ObjectivesTo use Population Health Management (PHM) methods to identify and characterise individuals at high-risk of severe COVID-19 for which shielding is required, for the purposes of managing ongoing health needs and mitigating potential shielding-induced harm.DesignIndividuals at ‘high risk’ of COVID-19 were identified using the published national ‘Shielded Patient List’ criteria. Individual-level information, including current chronic conditions, historical healthcare utilisation and demographic and socioeconomic status, was used for descriptive analyses of this group using PHM methods. Segmentation used k-prototypes cluster analysis.SettingA major healthcare system in the South West of England, for which linked primary, secondary, community and mental health data are available in a system-wide dataset. The study was performed at a time considered to be relatively early in the COVID-19 pandemic in the UK.Participants1 013 940 individuals from 78 contributing general practices.ResultsCompared with the groups considered at ‘low’ and ‘moderate’ risk (ie, eligible for the annual influenza vaccination), individuals at high risk were older (median age: 68 years (IQR: 55–77 years), cf 30 years (18–44 years) and 63 years (38–73 years), respectively), with more primary care/community contacts in the previous year (median contacts: 5 (2–10), cf 0 (0–2) and 2 (0–5)) and had a higher burden of comorbidity (median Charlson Score: 4 (3–6), cf 0 (0–0) and 2 (1–4)). Geospatial analyses revealed that 3.3% of rural and semi-rural residents were in the high-risk group compared with 2.91% of urban and inner-city residents (p<0.001). Segmentation uncovered six distinct clusters comprising the high-risk population, with key differentiation based on age and the presence of cancer, respiratory, and mental health conditions.ConclusionsPHM methods are useful in characterising the needs of individuals requiring shielding. Segmentation of the high-risk population identified groups with distinct characteristics that may benefit from a more tailored response from health and care providers and policy-makers.

show abstract

Section: Discussionsupporting

confidence: 53%

Population Health Management to identify and characterise ongoing health need for high-risk individuals shielded from COVID-19: a cross-sectional cohort study

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Algorithmic bias occurs when an algorithm compounds and amplifies the existing inequities in socioeconomic status, race, ethnic background, religion, gender, disability, or sexual disorientation during its application, which adversely impacts equity in health systems. The factors contributing to algorithm bias lack exact definitions and standards of fairness, inadequate contextual specificity, and the black-box nature of algorithms [27].…”

Section: Challenges and Biases In Aimentioning

confidence: 99%

Is Artificial Intelligence the New Friend for Radiologists? A Review Article

Gampala¹,

Vankeshwaram²,

Gadula³

2020

Cureus

View full text Add to dashboard Cite

Artificial intelligence (AI) is a path-breaking advancement for many industries, including the health care sector. The expeditious development of information technology and data processing has led to the formation of recent tools known as artificial intelligence. Radiology has been a portal for medical technological advancements, and AI will likely be no dissimilar. Radiology is the platform for many technological advances in the medical field; AI can undoubtedly impact every step of a radiologist's workflow. AI can simplify every activity like ordering and scheduling, protocoling and acquisition, image interpretation, reporting, communication, and billing. AI has eminent potential to augment efficiency and accuracy throughout radiology, but it also possesses inherent drawbacks and biases. We collected studies that were published in the past five years using PubMed as our database. We chose studies that were relevant to artificial intelligence in radiology. We mainly focused on the overview of AI in radiology, components included in the functioning of AI, AI assisting in the radiologists' workflow, ethical aspects of AI, challenges, and biases that AI experiencing together with some clinical applications of AI. Of all 33 studies, we found 15 articles discussed the overview and components of AI, five articles about AI affecting radiologist's workflow, five articles related to challenges and biases in AI, two articles discussed ethical aspects of AI, and six articles about practical implications of AI. We found out that the application of AI could make time-dependent tasks that can be performed effortlessly, permitting radiologists more time and opportunities to engage in patient care via increased time for consultation and development in imaging and extracting useful data from those images. AI could only be an aid to radiologists but will not replace a radiologist. Radiologists who use AI to their benefit, rather than to avoid it out of fear, might supersede those radiologists who do not. Substantial research should be done regarding the practical implications of AI algorithms for residents curriculum and the benefits of AI in radiology.

show abstract

“…"Opt-in" models, though lighter touch, may result in only the most proactive patients engaging. The inherent risk is that algorithms will only work well in these proactive populations, compounding inequities in regard to age, ethnicity, and biological sex 16 . In 2016 the UK's National Data Guardian concluded that "opt-out" models would be the most appropriate for collection and secondary use of National Health Service data 17 and from 2018, a national data opt-out was instituted.…”

Section: Compliance and Contractingmentioning

confidence: 99%

“Yes, but will it work for my patients?” Driving clinically relevant research with benchmark datasets

Panch

Pollard

Mattie³

et al. 2020

npj Digit. Med.

Self Cite

View full text Add to dashboard Cite

Benchmark datasets have a powerful normative influence: by determining how the real world is represented in data, they define which problems will first be solved by algorithms built using the datasets and, by extension, who these algorithms will work for. It is desirable for these datasets to serve four functions: (1) enabling the creation of clinically relevant algorithms; (2) facilitating like-forlike comparison of algorithmic performance; (3) ensuring reproducibility of algorithms; (4) asserting a normative influence on the clinical domains and diversity of patients that will potentially benefit from technological advances. Without benchmark datasets that satisfy these functions, it is impossible to address two perennial concerns of clinicians experienced in computational research: "the data scientists just go where the data is rather than where the needs are," and, "yes, but will this work for my patients?" If algorithms are to be developed and applied for the care of patients, then it is prudent for the research community to create benchmark datasets proactively, across specialties. As yet, best practice in this area has not been defined. Broadly speaking, efforts will include design of the dataset; compliance and contracting issues relating to the sharing of sensitive data; enabling access and reuse; and planning for translation of algorithms to the clinical environment. If a deliberate and systematic approach is not followed, not only will the considerable benefits of clinical algorithms fail to be realized, but the potential harms may be regressively incurred across existing gradients of social inequity.

show abstract

Artificial intelligence and algorithmic bias: implications for health systems

Cited by 247 publications

References 7 publications

Population Health Management to identify and characterise ongoing health need for high-risk individuals shielded from COVID-19: a cross-sectional cohort study

Population Health Management to identify and characterise ongoing health need for high-risk individuals shielded from COVID-19: a cross-sectional cohort study

Is Artificial Intelligence the New Friend for Radiologists? A Review Article

“Yes, but will it work for my patients?” Driving clinically relevant research with benchmark datasets

Contact Info

Product

Resources

About