Probability binning comparison: a metric for quantitating multivariate distribution differences

Abstract

Background: While several algorithms for the comparison of univariate distributions arising from flow cytometric analyses have been developed and studied for many years, algorithms for comparing multivariate distributions remain elusive. Such algorithms could be useful for comparing differences between samples based on several independent measurements, rather than differences based on any single measurement. It is conceivable that distributions could be completely distinct in multivariate space, but unresolvable in any combination of univariate histograms. Multivariate comparisons could also be useful for providing feedback about instrument stability, when only subtle changes in measurements are occurring. Methods: We apply a variant of Probability Binning, described in the accompanying article, to multidimensional data. In this approach, hyper-rectangles of n dimensions (where n is the number of measurements being compared) comprise the bins used for the chi-squared statistic. These hyper-dimensional bins are constructed such that the control sample has the same number of events in each bin; the bins are then applied to the test samples for chi-squared calculations. Results: Using a Monte-Carlo simulation, we determined the distribution of chi-squared values obtained by comparing sets of events from the same distribution; this distri-bution of chi-squared values was identical as for the univariate algorithm. Hence, the same formulae can be used to construct a metric, analogous to a t-score, that estimates the probability with which distributions are distinct. As for univariate comparisons, this metric scales with the difference between two distributions, and can be used to rank samples according to similarity to a control. We apply the algorithm to multivariate immunophenotyping data, and demonstrate that it can be used to discriminate distinct samples and to rank samples according to a biologically-meaningful difference. Conclusion: Probability binning, as shown here, provides a useful metric for determining the probability with which two or more multivariate distributions represent distinct sets of data. The metric can be used to identify the similarity or dissimilarity of samples. Finally, as demonstrated in the accompanying paper, the algorithm can be used to gate on events in one sample that are different from a control sample, even if those events cannot be distinguished on the basis of any combination of univariate or bivariate displays. Cytometry 45:47-55, 2001.

Identification of organ‐specific T cell populations by analysis of multiparameter flow cytometry data using DNA‐chip analysis software

“…However, it turns out to be a major task with an increasing number of populations and experimental groups, and becomes an even more difficult undertaking, if there is uncertainty about which populations have to be compared and which might be irrelevant. Recently automated clustering algorithms have been implemented (1,(3)(4)(5), which can be used to identify populations significantly different from a control sample. However, these algorithms reach their limitations, if replicates are to be compared.…”

“…After comparison to the appropriate staining controls (30) antigen-positive and -negative cells can be discriminated manually and further analysis is completed using statistics software. In contrast to previously published methods (3)(4)(5) statistics is performed on a different level, i.e. not on primary data, but on secondary data representing whole populations.…”

“…Therefore, in most multiparameter flow cytometry experiments reported to date, potentially important information might have been ignored. Progress towards solving this problem has been made by automated clustering algorithms (1), with which populations of interest can be identified in the multidimensional flow cytometric space (3)(4)(5). Using the probability binning approach by Roederer et al (3,4) individual samples can be compared (6), however, the comparison of replicates of multiple experimental groups still remains a major problem.…”

Mapping cell populations in flow cytometry data for cross‐sample comparison using the Friedman–Rafsky test statistic as a distance measure

“…Probability binning 12 is able to compare multivariate distributions between FCM samples but it remains unclear how it can be adapted to compare population‐level data as cell populations frequently shift expression distributions across samples. Finak et al 13 compared sample‐level variability in cell population marker expression among fluorescent channel transformation methods (e.g., bi‐exponential or generalized Box‐Cox).…”

Analysis of flow cytometry data using an automatic processing tool

“…The APT was specifically designed for situations where objective standards for the definition of cutoffs and response levels are unavailable. Previous work has developed mathematical tools for the objective characterization of flow cytometry data (4)(5)(6)(7)(8), but without providing a framework for their implementation. A recent automated approach used a multivariate neural network approach (9), which is radically different from the techniques used for manual processing.…”