Bias in algorithms of AI systems developed for COVID-19: A scoping review

Abstract

To analyze which ethically relevant biases have been identified by academic literature in artificial intelligence (AI) algorithms developed either for patient risk prediction and triage, or for contact tracing to deal with the COVID-19 pandemic. Additionally, to specifically investigate whether the role of social determinants of health (SDOH) have been considered in these AI developments or not. We conducted a scoping review of the literature, which covered publications from March 2020 to April 2021. ​Studies mentioning biases on AI algorithms developed for contact tracing and medical triage or risk prediction regarding COVID-19 were included. From 1054 identified articles, 20 studies were finally included. We propose a typology of biases identified in the literature based on bias, limitations and other ethical issues in both areas of analysis. Results on health disparities and SDOH were classified into five categories: racial disparities, biased data, socio-economic disparities, unequal accessibility and workforce, and information communication. SDOH needs to be considered in the clinical context, where they still seem underestimated. Epidemiological conditions depend on geographic location, so the use of local data in studies to develop international solutions may increase some biases. Gender bias was not specifically addressed in the articles included. The main biases are related to data collection and management. Ethical problems related to privacy, consent, and lack of regulation have been identified in contact tracing while some bias-related health inequalities have been highlighted. There is a need for further research focusing on SDOH and these specific AI apps.

Patient-centered pharmacovigilance: priority actions from the inherited bleeding disorders community

“… 72 , 78 The integration of AI into any healthcare process must be done with explicit awareness of, and efforts to eliminate, algorithmic bias that may result from the design of the algorithm, previous data collection, coding, and selection. 79 , 80 When queried about their comfort level with researchers or the Medicines and Healthcare products Regulatory Agency (MHRA), which monitors and regulates postmarketing drug safety in the UK, using content posted to the aforementioned online platform to help monitor side effects, over 90% of respondents were favorable to the idea. 69 They did, however, express concerns about data privacy, ethics, and the ability of such techniques to truly understand the nuanced and personal content being shared.…”

The Impact and Implication of Artificial Intelligence on Thematic Healthcare and Quality of Life

“…However, companies such as Amazon have raised their concerns regarding the fight by mounting numerous public campaigns criticizing it. Delgado et al (2022) demonstrate the extent of bias in AI-based technologies by studying the algorithms of the AI systems developed to help in the fight against COVID-19. These researchers identified racial disparities as one of the biases of the AI systems designed for digital contact tracing (DCT) and triage for COVID-19.…”

“…This false sense makes the stakeholders fail to design and implement mitigation strategies and hinders the adoption of other measures and tools with the substantial potential of enhancing patients' outcomes. Regarding socioeconomic disparity, Delgado et al (2022) state that DCT is effective only if the patient has adequate personal wealth and can afford to stay at home for as long as necessary.…”

Ethical assessments and mitigation strategies for biases in AI-systems used during the COVID-19 pandemic

“…In previous research (Delgado et al, 2022), we found that AI systems have been mainly employed in triage and patient risk prediction and CTApps. Even though the implementation of AI has offered many benefits, the huge amount of data involved and the rapid rate of technological implementation have generated important ethical issues related to the appearance of biases in these areas of implementation.…”