A Mathematical Framework for Combining Decisions of Multiple Experts toward Accurate and Remote Diagnosis of Malaria Using Tele-Microscopy

Abstract

We propose a methodology for digitally fusing diagnostic decisions made by multiple medical experts in order to improve accuracy of diagnosis. Toward this goal, we report an experimental study involving nine experts, where each one was given more than 8,000 digital microscopic images of individual human red blood cells and asked to identify malaria infected cells. The results of this experiment reveal that even highly trained medical experts are not always self-consistent in their diagnostic decisions and that there exists a fair level of disagreement among experts, even for binary decisions (i.e., infected vs. uninfected). To tackle this general medical diagnosis problem, we propose a probabilistic algorithm to fuse the decisions made by trained medical experts to robustly achieve higher levels of accuracy when compared to individual experts making such decisions. By modelling the decisions of experts as a three component mixture model and solving for the underlying parameters using the Expectation Maximisation algorithm, we demonstrate the efficacy of our approach which significantly improves the overall diagnostic accuracy of malaria infected cells. Additionally, we present a mathematical framework for performing ‘slide-level’ diagnosis by using individual ‘cell-level’ diagnosis data, shedding more light on the statistical rules that should govern the routine practice in examination of e.g., thin blood smear samples. This framework could be generalized for various other tele-pathology needs, and can be used by trained experts within an efficient tele-medicine platform.

A game-based platform for crowd-sourcing biomedical image diagnosis and standardized remote training and education of diagnosticians

“…The amount of Figure 7. Expert label agreement per individual RBC images for >8500 RBC images 21 . When comparing the expert diagnoses for each RBC image, the agreement per label decreases significantly.…”

“…Figure 8. Expert self-inconsistency for each RBC image within each label category for >8500 RBC images 21 . For example, expert 1 changed their diagnosis for 1.8%, 1.3%, and 1.6% of cell images initially labeled questionable, positive, or negative out of 3 diagnoses taken from 90 degree rotated cell images.…”

“…Following the success of these systems, we developed a game-based crowdsourcing platform called BioGames 14 , [19][20][21][22] for distributed biomedical image analysis, accessible via the Internet browser or through smartphone apps (see Figure 1). We demonstrated the system's capability using a game where users could diagnose individual red blood cell (RBC) images for the presence of malaria in a study at UCLA, and subsequently released the platform to the general public through a website (http://biogames.ee.ucla.edu) and mobile apps (see Figure 1b).…”

“…Left: Forward 3-label mixture model describing the application of each j th expert's error probabilities P j (x|I) onto each n th cell image to generate expert responses x nj for all N cell images and all M experts21 . Right: Decoding model utilizing an Expectation Maximization (EM) algorithm35 to generate Maximum Likelihood labels for each cell image, where each x nj acts as the observed variables to find the true image labels I n with parameters P j (x|I).…”

Red blood cell as an adaptive optofluidic microlens

“…In particular, several blood diseases involve modification in the morphology of such cells, that is, sickle-cell disease, malaria and spherocytosis [1][2][3] . For these reasons, many non-invasive optical imaging systems have been developed for the investigation of RBC morphology as a diagnostic clinical factor [4][5][6][7][8] . In particular, quantitative phasecontrast methods have been explored for three-dimensional (3D) shape retrieving of live and unlabelled erythrocytes measuring volumes, refractive indexes and morphological parameters [9][10][11][12] .…”