Does Machine Learning Automate Moral Hazard and Error?

Abstract

Machine learning tools are beginning to be deployed en masse in health care. While the statistical underpinnings of these techniques have been questioned with regard to causality and stability, we highlight a different concern here, relating to measurement issues. A characteristic feature of health data, unlike other applications of machine learning, is that neither y nor x is measured perfectly. Far from a minor nuance, this can undermine the power of machine learning algorithms to drive change in the health care system--and indeed, can cause them to reproduce and even magnify existing errors in human judgment.

Machine learning for radiation outcome modeling and prediction

“…For instance, an ANN (a noninterpretable algorithm) that was developed to triage patients with pneumonia for hospital discharge was found to inadvertently label asthmatic patients as low risk . Colon cancer screening or abnormal breast finding were found to be highly correlated with the risk of having a stroke with no apparent clinical justification . A CNN was found to be using the hospital system, department or the label “portable” to improve their prediction of pneumonia while disregarding the true findings in the image .…”

Lost in Thought — The Limits of the Human Mind and the Future of Medicine

“…Ignoring these facts will automate and even magnify problems in our current health system. 5 Noticing and undoing these problems requires a deep familiarity with clinical decisions and the data they produce — a reality that highlights the importance of viewing algorithms as thinking partners, rather than replacements, for doctors.…”

A contemporary review of machine learning in otolaryngology–head and neck surgery

“…Measurements of lesion dimensions on radiology images, serum marker concentrations, and disease‐specific survival in a population sample all carry a margin of error. Concerns have been raised on the potential for machine‐learning implementations to do harm by propagating error from imperfect measures . Similarly, one often‐flouted concern is that we are training machines to take over human judgement, decision‐making, and intuition .…”