High Tech, High Risk: Tech Ethics Lessons for the COVID-19 Pandemic Response

Abstract

The COVID-19 pandemic has, in a matter of a few short months, drastically reshaped society around the world. Because of the growing perception of machine learning as a technology capable of addressing large problems at scale, machine learning applications have been seen as desirable interventions in mitigating the risks of the pandemic disease. However, machine learning, like many tools of technocratic governance, is deeply implicated in the social production and distribution of risk and the role of machine learning in the production of risk must be considered as engineers and other technologists develop tools for the current crisis. This paper describes the coupling of machine learning and the social production of risk, generally, and in pandemic responses specifically. It goes on to describe the role of risk management in the effort to institutionalize ethics in the technology industry and how such efforts can benefit from a deeper understanding of the social production of risk through machine learning.

Contact tracing apps for the COVID-19 pandemic: a systematic literature review of challenges and future directions for neo-liberal societies

“…Therefore, they may be forced to quarantine simply because they have been in close proximity with others in the same social group although they may not be at high risk of infection. The lack of smartphones and internet access, as well as the share of informal employment all come together to disproportionately impact the low-income communities which continue to drive the health divide rooted in social status and economic differences even further [ 68 , 69 ]. Primary studies [ 64 , 70 – 72 ], discuss the risk of data collected through contact tracing apps by public health authorities and governments can be used not just for epidemiological studies and surveillance but also for behavioural profiling of a population.…”

“…Therefore, they may be forced to quarantine simply because they have been in close proximity with others in the same social group although they may not be at high risk of infection. The lack of smartphones and internet access, as well as the share of informal employment all come together to disproportionately impact the low-income communities which continue to drive the health divide rooted in social status and economic differences even further [ 68 , 69 ].…”

A Proposal for Identifying and Managing Bias in Artificial Intelligence

“…The reviewed literature suggests that the expansion of AI into many aspects of public life requires extending our view from a mainly technical perspective to one that considers AI within the social system it operates [3,18,19,31,34,40,41,43,71,97,118,120,134]. Taking social factors into consideration is necessary for achieving trustworthy AI, and can enable a broader understanding of AI impacts and the key decisions that happen throughout, and beyond, the AI lifecycle -such as whether technology is even a solution to a given task or problem [11,49].…”

“…Operational settings and unknown impacts Current assumptions in AI development often revolve around the idea of technological solutionism -the perception that technology will lead to only positive solutions. This perception, often combined with a singular focus on tool optimization, can be at odds with operational scenarios, increasing the difficulty for the practitioners who have to make sense of tool outputoften in high stakes settings [96]. What seems like a good idea for how a given dataset can be utilized in a specific use case might be perceived differently by the systems' end users or those affected by the systems' decisions.…”

“…Identifying and addressing potential biases early in the problem formulation process is an important step in this process. It is also complicated by the role of power and decision making [96]. A consistent theme from the literature is the benefit of engaging a variety of stakeholders and maintaining diversity along social lines where bias is a concern (racial diversity, gender diversity, age diversity, diversity of physical ability) [32].…”