Accounting for diversity in AI for medicine

Fosch‐Villaronga, Eduard; Drukarch, Hadassah; Khanna, Pranav; Verhoef, Tessa; Custers, Bart

doi:10.1016/j.clsr.2022.105735

Cited by 15 publications

(8 citation statements)

References 88 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the anticipated utility of proactive AI-DSSs must be carefully balanced against the predominant viewpoint that automation of decision-making in the nursing process should be avoided (prerequisite 2), that AI-DSSs might only be introduced in practice through incremental steps that are aligned with users' evolving trust in, and experience with the use of these systems (prerequisite 3), and that vigilance is required to prevent that caregivers become overly reliant on AI-DSSs and are led astray towards unsuitable care strategies (see also [44,45]). In this regard, our findings highlight the importance of actively counteracting bias and narrow perspectives during both the design and use of AI-DSSs (prerequisite 5) (see also [46][47][48]), and setting up human-centric learning loops through which caregivers can actively contribute to meaningful and responsible design, implementation and use of AI-DSSs (prerequisite 6) [48,49]. These findings resonate with Hindocha and Cosmin Badea [38], who argue that care professionals will be integral to the responsible design, deployment and use of AI in healthcare, as they can act as the moral exemplar for the virtuous machine.…”

Section: Implications For Research and Practicementioning

confidence: 69%

Artificial intelligence-assisted decision-making in long-term care: a qualitative study on opportunities and prerequisites for responsible innovation (Preprint)

Lukkien,

Stolwijk,

Ipakchian Askari

et al. 2024

Preprint

View full text Add to dashboard Cite

BACKGROUND While use of artificial intelligence (AI)-based technologies such as decision-support systems (AI-DSSs) could help sustaining and improving the quality and efficiency of care, their deployment also creates ethical and social challenges. In recent years, there has been a growing prevalence of high-level guidelines and frameworks to provide guidance on responsible AI innovation. However, few studies specify how AI-based technologies such as AI-DSSs can be responsibly embedded in specific contexts such as the nursing process in the long-term care (LTC) for older adults. OBJECTIVE Opportunities and prerequisites for responsible AI-assisted decision-making in the nursing process were explored from the perspectives of nurses and other professional stakeholders in LTC. METHODS Semi-structured interviews were conducted with 24 care professionals in Dutch LTC, including nurses, care coordinators, data specialists and care centralists. Two imaginary scenarios about the future use of AI-DSSs were developed beforehand and used to enable participants to articulate their expectations regarding the opportunities and risks of AI-assisted decision-making. After first openly discussing opportunities and possible risks associated with both scenarios, six high-level principles for responsible AI were used as probing themes to evoke further consideration on risks of using AI-DSSs in LTC. Further, participants were asked to brainstorm about possible strategies and actions in the design, implementation and use of AI-DSSs to address or mitigate the mentioned risks. A thematic analysis was carried out to identify opportunities and prerequisites for responsible innovation in this area. RESULTS Professionals’ stance towards the use of AI-DSSs is not a matter of purely positive or negative expectations, but rather a nuanced interplay of positive and negative elements that lead to a weighed perception of opportunities and prerequisites for responsible AI-assisted decision-making. Both opportunities and risks were identified in relation to early identification of care needs, guidance in devising care strategies, shared decision-making, and caregivers’ workload and work experience. To optimally balance opportunities and risks of AI-assisted decision-making, seven categories of prerequisites for responsible AI-assisted decision-making in the nursing process were identified: (1) regular deliberation on data collection, (2) a balanced proactive nature of AI-DSSs, (3) incremental advancements aligned with trust and experience, (4) customization for all user groups including clients and caregivers, (5) measures to counteract bias and narrow perspectives, (6) human-centric learning loops, and (7) routinization of using AI-DSSs. CONCLUSIONS Opportunities of AI-assisted decision-making in the nursing process could turn into drawbacks, depending on the specific shaping of the design and the deployment of AI-DSSs. Therefore, we recommend viewing the responsible use of AI-DSSs as a balancing act. Moreover, given the interrelatedness of the identified prerequisites, we call for various actors, including developers and users of AI-DSSs, to cohesively address different factors important to the responsible embedding of AI-DSSs in practice.

show abstract

Section: Implications For Research and Practicementioning

confidence: 69%

Artificial intelligence-assisted decision-making in long-term care: a qualitative study on opportunities and prerequisites for responsible innovation (Preprint)

Lukkien,

Stolwijk,

Ipakchian Askari

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…They emphasize the importance of accounting for gender and sex differences in the development of medical algorithms, warning that the neglect of these factors could lead to misdiagnoses and potential discrimination. This research underscores the need to incorporate diversity and inclusion considerations into AI developments in healthcare, to prevent exacerbating existing biases or creating new ones [7].…”

Section: Ethical Considerations In Aimentioning

confidence: 86%

Artificial Intelligence Ethics and Challenges in Healthcare Applications: A Comprehensive Review in the Context of the European GDPR Mandate

Mohammad Amini,

Jesus,

Fanaei Sheikholeslami

et al. 2023

MAKE

View full text Add to dashboard Cite

This study examines the ethical issues surrounding the use of Artificial Intelligence (AI) in healthcare, specifically nursing, under the European General Data Protection Regulation (GDPR). The analysis delves into how GDPR applies to healthcare AI projects, encompassing data collection and decision-making stages, to reveal the ethical implications at each step. A comprehensive review of the literature categorizes research investigations into three main categories: Ethical Considerations in AI; Practical Challenges and Solutions in AI Integration; and Legal and Policy Implications in AI. The analysis uncovers a significant research deficit in this field, with a particular focus on data owner rights and AI ethics within GDPR compliance. To address this gap, the study proposes new case studies that emphasize the importance of comprehending data owner rights and establishing ethical norms for AI use in medical applications, especially in nursing. This review makes a valuable contribution to the AI ethics debate and assists nursing and healthcare professionals in developing ethical AI practices. The insights provided help stakeholders navigate the intricate terrain of data protection, ethical considerations, and regulatory compliance in AI-driven healthcare. Lastly, the study introduces a case study of a real AI health-tech project named SENSOMATT, spotlighting GDPR and privacy issues.

show abstract

“…Relevantly, the algorithms of AI can perpetuate or decrease the sex and gender bias depending on how they are developed, without removing bias and confounding factors or integrating sex and gender differences [56]. Unfortunately, despite the influences of sex and gender on health and medicine, most biomedical AI technologies do not consider sex and gender [56,57]. In addition, fewer women are included in AI studies, including those focused on digital biomarkers.…”

Section: Technologies 3d Scanning and Artificial Intelligence (Ai) Th...mentioning

confidence: 99%

The Appropriateness of Medical Devices Is Strongly Influenced by Sex and Gender

Campesi,

Franconi,

Serra

2024

Life

View full text Add to dashboard Cite

Until now, research has been performed mainly in men, with a low recruitment of women; consequentially, biological, physiological, and physio-pathological mechanisms are less understood in women. Obviously, without data obtained on women, it is impossible to apply the results of research appropriately to women. This issue also applies to medical devices (MDs), and numerous problems linked to scarce pre-market research and clinical trials on MDs were evidenced after their introduction to the market. Globally, some MDs are less efficient in women than in men and sometimes MDs are less safe for women than men, although recently there has been a small but significant decrease in the sex and gender gap. As an example, cardiac resynchronization defibrillators seem to produce more beneficial effects in women than in men. It is also important to remember that MDs can impact the health of healthcare providers and this could occur in a sex- and gender-dependent manner. Recently, MDs’ complexity is rising, and to ensure their appropriate use they must have a sex–gender-sensitive approach. Unfortunately, the majority of physicians, healthcare providers, and developers of MDs still believe that the human population is only constituted by men. Therefore, to overcome the gender gap, a real collaboration between the inventors of MDs, health researchers, and health providers should be established to test MDs in female and male tissues, animals, and women.

show abstract

Accounting for diversity in AI for medicine

Cited by 15 publications

References 88 publications

Artificial intelligence-assisted decision-making in long-term care: a qualitative study on opportunities and prerequisites for responsible innovation (Preprint)

Artificial intelligence-assisted decision-making in long-term care: a qualitative study on opportunities and prerequisites for responsible innovation (Preprint)

Artificial Intelligence Ethics and Challenges in Healthcare Applications: A Comprehensive Review in the Context of the European GDPR Mandate

The Appropriateness of Medical Devices Is Strongly Influenced by Sex and Gender

Contact Info

Product

Resources

About