Setting objective thresholds for rare event detection in flow cytometry

Abstract

The accurate identification of rare antigen-specific cytokine positive cells from peripheral blood mononuclear cells (PBMC) after antigenic stimulation in an intracellular staining (ICS) flow cytometry assay is challenging, as cytokine positive events may be fairly diffusely distributed and lack an obvious separation from the negative population. Traditionally, the approach by flow operators has been to manually set a positivity threshold to partition events into cytokine-positive and cytokine-negative. This approach suffers from subjectivity and inconsistency across different flow operators. The use of statistical clustering methods does not remove the need to find an objective threshold between between positive and negative events since consistent identification of rare event subsets is highly challenging for automated algorithms, especially when there is distributional overlap between the positive and negative events (“smear”). We present a new approach, based on the Fβ measure, that is similar to manual thresholding in providing a hard cutoff, but has the advantage of being determined objectively. The performance of this algorithm is compared with results obtained by expert visual gating. Several ICS data sets from the External Quality Assurance Program Oversight Laboratory (EQAPOL) proficiency program were used to make the comparisons. We first show that visually determined thresholds are difficult to reproduce and pose a problem when comparing results across operators or laboratories, as well as problems that occur with the use of commonly employed clustering algorithms. In contrast, a single parameterization for the Fβ method performs consistently across different centers, samples, and instruments because it optimizes the precision/recall tradeoff by using both negative and positive controls.

High-speed automatic characterization of rare events in flow cytometric data

“…Some of these have also involved hierarchical and multi-level models [8,10,17]. Often, such methods were designed with the aim of detecting both known as well as rare cell clusters [4,[18][19][20] in an automated manner. Other studies have developed fast clustering algorithms with the aim of handling large cytometric data [21][22][23].…”

Conjunctival HLA-DR Expression and Its Association With Symptoms and Signs in the DREAM Study

“…Although the variation in average HLA-DR% between other studies of DED could be attributed to differences in inclusion criteria, the biggest contributor is likely the lack of standardized methods for flow cytometry instrumentation, data acquisition, and analysis. This has been acknowledged as an important issue in diseases where immunophenotyping is used for clinical diagnosis, the detection of rare phenotypes, and monitoring treatment response 48,49. Efforts to standardize sampling, flow cytometer performance between runs, and equalization of data obtained from different flow cytometers are already under way for other diseases; the European Union-supported EuroFlow Consortium and Human Immune Phenotyping Consortium are notable examples 50,51.…”

Toward development of a comprehensive external quality assurance program for polyfunctional intracellular cytokine staining assays

“…Manual analysis is limited by conventional analysis tools that utilize sequential two dimensional gating and local operator expertise. To overcome this issue the Flow Cytometry EQAP includes the development and implementation of automated analysis tools Richards et al (in this issue). The Flow Cytometry EQAP partnered with Flow Cytometry: Critical Assessment of Population Identification Methods (FlowCAP) to facilitate development and implementation of an optimized automated analysis approach for accurate rare event quantification.…”