Probabilistic change detection and visualization methods for the assessment of temporal stability in biomedical data quality

Abstract

Knowledge discovery on biomedical data can be based on on-line, data-stream analyses, or using retrospective, timestamped, off-line datasets. In both cases, changes in the processes that generate data or in their quality features through time may hinder either the knowledge discovery process or the generalization of past knowledge. These problems can be seen as a lack of data temporal stability. This work establishes the temporal stability as a data quality dimension and proposes new methods for its assessment based on a probabilistic framework. Concretely, methods are proposed for 1) monitoring changes, and 2) characterizing changes, trends and detecting temporal subgroups. First, a probabilistic change detection algorithm is proposed based on the Statistical Process Control of the posterior Beta distribution of the Jensen-Shannon distance, with a memoryless forgetting mechanism. This algorithm (PDF-SPC) classifies the degree of current change in three states: In-Control, Warning, and Out-of-Control. Second, a novel method is proposed to visualize and characterize the temporal changes of data based on the projection of a non-parametric information-geometric statistical manifold of time windows. This projection facilitates the exploration of temporal trends using the proposed IGT-plot and, by means of unsupervised learning methods, discovering conceptually-related temporal subgroups. Methods are evaluated using real and simulated data based on the National Hospital Discharge Survey (NHDS) dataset.

Applying probabilistic temporal and multisite data quality control methods to a public health mortality registry in Spain: a systematic approach to quality control of repositories

“…The methods used in the present study fall into two groups, namely those for assessing multisource variability [18] and those for assessing temporal variability. [19] The methods are based on the comparison of probability distributions of the variables among different sources or over different periods of time. The comparisons are made by calculating the information-theoretic probabilistic distances between pairs of distributions, in concrete terms, we use the Jensen-Shannon distance (JSD), a symmetrized and smoothed version of the Kullback-Leibler divergence.…”

“…Multi-source or temporal variability, if unmanaged, may lead to inaccurate or irreproducible results [3,18,19] or even to invalid results. [11] The reuse of data in multi-site repositories for population studies, clinical trials, or data mining rests on the assumption that the data distributions are to some degree concordant irrespective of the source of data or of the time over which the data have been collected and therefore allows generalizable conclusions to be drawn from the data.…”

“…[12] Meeting the requirements mentioned above, we developed two sets of methods for both multi-source and temporal variability assessment. [18,19] These methods allow variability to be measured and visually explored, by comparing the probability distributions of the data using information-theoretic metrics. These methods have been evaluated earlier using simulated problems as well as real registries, including the UCI Heart Disease public dataset [26] and the US NHDS data.…”

Data-driven discovery of changes in clinical code usage over time: a case-study on changes in cardiovascular disease recording in two English electronic health records databases (2001-2015)

“…was not certified by peer review) (which The copyright holder for this preprint this version posted September 16, 2019. . https://doi.org/10.1101/19006098 doi: medRxiv preprint A PREPRINT -SEPTEMBER 5, 2019 framework can easily be extended to allow for a more formal statistical process control (see Sáez et al (2015 and [17,19] for guidance).…”

Subgroup discovery

“…Larger search spaces, like those encountered for numerical data, (complex) multi‐relational datasets, for example, encountered in social networks, or spatiotemporal data require efficient algorithms that can handle those different types of data, e.g., Refs . Also combinations of such different data characteristics, for example, temporal pattern mining for event detection, or temporal subgroup analytics provide further challenges, especially considering sophisticated exceptional model classes in that area.…”