Evolution and impact of bias in human and machine learning algorithm interaction

Abstract

Traditionally, machine learning algorithms relied on reliable labels from experts to build predictions. More recently however, algorithms have been receiving data from the general population in the form of labeling, annotations, etc. The result is that algorithms are subject to bias that is born from ingesting unchecked information, such as biased samples and biased labels. Furthermore, people and algorithms are increasingly engaged in interactive processes wherein neither the human nor the algorithms receive unbiased data. Algorithms can also make biased predictions, leading to what is now known as algorithmic bias. On the other hand, human's reaction to the output of machine learning methods with algorithmic bias worsen the situations by making decision based on biased information, which will probably be consumed by algorithms later. Some recent research has focused on the ethical and moral implication of machine learning algorithmic bias on society. However, most research has so far treated algorithmic bias as a static factor, which fails to capture the dynamic and iterative properties of bias. We argue that algorithmic bias interacts with humans in an iterative manner, which has a long-term effect on algorithms' performance. For this purpose, we present an iterated-learning framework that is inspired from human language evolution to study the interaction between machine learning algorithms and humans. Our goal is to study two sources of bias that interact: the process by which people select information to label (human action); and the process by which an algorithm selects the subset of information to present to people (iterated algorithmic bias mode). We investigate three forms of iterated algorithmic bias (personalization filter, active learning, and random) and how they affect the performance of machine learning algorithms by formulating research questions about the impact of each type of bias. Based on statistical analyses of the results of several controlled experiments, we found that the three different iterated bias modes, as well as initial training data class imbalance and human action, do affect the models learned by machine learning algorithms. We also found that iterated filter bias, which is prominent in personalized user interfaces, can lead to more inequality in estimated relevance and to a limited human ability to discover relevant data. Our findings indicate that the relevance blind spot (items from the testing set whose predicted relevance probability is less than 0.5 and who thus risk being hidden from humans) amounted to 4% of all relevant items when using a content-based filter

Interpretability of Machine Learning Solutions in Public Healthcare: The CRISP-ML Approach

“…These developments increased the pressure on creation of frameworks and methodologies, which can ensure sufficient interpretability of ML solutions. In healthcare, such pressure is amplified by the nature of the interactive processes, wherein neither humans nor the algorithms operate with unbiased data (Sun et al, 2020 ).…”

Bias and unfairness in machine learning models: a systematic literature review

“…The paper [44] studied filter bias, active learning bias, and random baseline bias in the field of information filtering algorithms, and classified them as capable of affecting their performance. Furthermore, the authors proposed a framework to analyze bias in these systems and were able to conclude that filter bias, prominent in personalized user interfaces, can limit a user's ability to discover relevant information that could be presented to them.…”

Artificial intelligence for diagnosis and prognosis in neuroimaging for dementia; a systematic review

“…The over-reliance on a single dataset such as ADNI from one country introduces potential ethnic and socio-economic biases to models that may hamper generalisation, an issue that has been specifically raised in the ADNI dataset (Mendelson et al 2017). Concerns have been raised more generally about bias in AI/ML models (Sun, Nasraoui, and Shafto 2020), including in the context of health applications (Parikh, Teeple, and Navathe 2019). This is of particular concern in marginalised ethinic groups who have poorer health indicators in general (Williams 2012), and who may miss out on access to health services due to sociodemographic, cultural or religious beliefs (Obermeyer et al 2019), including dementia services (Razai et al 2021;Mukadam, Cooper, and Livingston 2011).…”