Medical AI: is trust really the issue?

Mainz, Jakob Thrane

doi:10.1136/jme-2023-109414

Cited by 2 publications

(2 citation statements)

References 4 publications

(11 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, we respond to the "so-what objection"(Mainz, 2023) by rejecting the claim that it is trivial to point out that AI cannot be trustworthy. Simply put, what we say matters: calling AI trustworthy, when it is not, propagates misinformation and engenders miscommunication that could result in mistrust in science (National Academies of Sciences, 2017).…”

mentioning

confidence: 94%

“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”

Dorsch,

Deroy

2024

Philos. Technol.

View full text Add to dashboard Cite

Many policies and ethical guidelines recommend developing “trustworthy AI”. We argue that developing morally trustworthy AI is not only unethical, as it promotes trust in an entity that cannot be trustworthy, but it is also unnecessary for optimal calibration. Instead, we show that reliability, exclusive of moral trust, entails the appropriate normative constraints that enable optimal calibration and mitigate the vulnerability that arises in high-stakes hybrid decision-making environments, without also demanding, as moral trust would, the anthropomorphization of AI and thus epistemically dubious behavior. The normative demands of reliability for inter-agential action are argued to be met by an analogue to procedural metacognitive competence (i.e., the ability to evaluate the quality of one’s own informational states to regulate subsequent action). Drawing on recent empirical findings that suggest providing reliability scores (e.g., F1-scores) to human decision-makers improves calibration in the AI system, we argue that reliability scores provide a good index of competence and enable humans to determine how much they wish to rely on the system.

show abstract

mentioning

confidence: 94%

“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”

Dorsch,

Deroy

2024

Philos. Technol.

View full text Add to dashboard Cite

show abstract

“…Wang et al [20] proposed that the data used for training AI may suffer from issues such as incompleteness, inconsistency, and inaccuracy, and considerations should also be given to data accessibility and privacy security. AI sharing patient data without the patient's explicit consent will leave the patient's privacy unprotected [21,22]. Therefore AI techniques are not suitable for application in uncertain, variable, critical, and complex environments [23], and the importance of AI research in healthcare is highlighted [24].…”

Section: Introductionmentioning

confidence: 99%

The DIKWP (Data, Information, Knowledge, Wisdom, Purpose) Revolution: A New Horizon in Medical Dispute Resolution

Mei,

Duan

2024

Applied Sciences

View full text Add to dashboard Cite

The doctor–patient relationship has received widespread attention as a significant global issue affecting people’s livelihoods. In clinical practice within the medical field, applying existing artificial intelligence (AI) technology presents issues such as uncontrollability, inconsistency, and lack of self-explanation capabilities, even raising concerns about ethics and morality. To address the problem of doctor–patient interaction differences arising from the doctor–patient diagnosis and treatment, we collected the textual content of doctor–patient dialogues in outpatient clinics of local first-class hospitals. We utilized case scenario analysis, starting from two specific cases: multi-patient visits with the same doctor and multi-doctor interaction differences with the same patient. By capturing the external interactions and the internal thought processes, we unify the external expressions and internal subjective cognition in doctor–patient interactions into interactions between data, information, knowledge, wisdom, and purpose (DIKWP) models. We propose a DIKWP semantic model for the doctor–patient interactions on both sides, including a DIKWP content model and a DIKWP cognitive model, to achieve transparency throughout the entire doctor–patient interaction process. We semantically–bidirectionally map the diagnostic discrepancy space to DIKWP uncertainty and utilize a purpose-driven DIKWP semantic fusion transformation technique to disambiguate the uncertainty problem. Finally, we select four traditional methods for qualitative and quantitative comparison with our proposed method. The results show that our method performs better in content and uncertainty handling. Overall, our proposed DIKWP semantic model for doctor–patient interaction processing breaks through the uncertainty limitations of natural language semantics in terms of interpretability, enhancing the transparency and interpretability of the medical process. It will help bridge the cognitive gap between doctors and patients, easing medical disputes.

show abstract

Medical AI: is trust really the issue?

Cited by 2 publications

References 4 publications

“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”

“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”

The DIKWP (Data, Information, Knowledge, Wisdom, Purpose) Revolution: A New Horizon in Medical Dispute Resolution

Contact Info

Product

Resources

About