Are Standard Diagnostic Test Characteristics Sufficient for the Assessment of Continual Patient Monitoring?

Abstract

Background. For diagnostic processes involving continual measurements from a single patient, conventional test characteristics, such as sensitivity and specificity, do not consider decision consistency, which might be a distinct, clinically relevant test characteristic. Objective. The authors investigated the performance of a decision-support classifier for the diagnosis of traumatic injury with blood loss, implemented with three different data-processing methods. For each method, they computed standard diagnostic test characteristics and novel metrics related to decision consistency and latency. Setting. Prehospital air ambulance transport. Patients. A total of 557 trauma patients. Design. Continually monitored vital-sign data from 279 patients (50%) were randomly selected for classifier development, and the remaining were used for testing. Three data-processing methods were evaluated over 16 min of patient monitoring: a 2-min moving window, time averaging, and postprocessing with the sequential probability ratio test (SPRT). Measurements. Sensitivity and specificity were computed. Consistency was quantified through cumulative counts of decision changes over time and the fraction of patients affected by false alarms. Latency was evaluated by the fraction of patients without a decision. Results. All 3 methods showed very similar final sensitivities and specificities. Yet, there were significant differences in terms of the fraction of patients affected by false alarms, decision changes through time, and latency. For instance, use of the SPRT led to a 75% reduction in the number of decision changes and a 36% reduction in the number of patients affected by false alarms, at the expense of 3% unresolved final decisions. Conclusion. The proposed metrics of decision consistency and decision latency provided additional information beyond what could be obtained from test sensitivity and specificity and are likely to be clinically relevant in some applications involving temporal decision making.

Automated Analysis of Vital Signs to Identify Patients With Substantial Bleeding Before Hospital Arrival

“…The SPRT (29) is a useful statistical technique for determining whether repeated measurement samples are consistent with one statistical distribution (e.g., a normal population) versus a second statistical distribution (e.g., an abnormal population). Thresholds for the SPRT were set as per Chen et al (10), where the SPRT was shown to reduce false alarms at the expense of some alarm latency.…”

“…Growing evidence shows that assessment of multiple vital signs together may be more effective than univariate approaches for detecting hemorrhagic hypovolemia (7,8). In addition, there have been encouraging reports of computational techniques (9,10) to account for the fact that high-acuity trauma patients demonstrate complex temporal fluctuations in their prehospital vital signs (11Y13), and to identify unreliable vital signs (14), because spurious measurements are so common (15Y18).…”

A platform for real-time acquisition and analysis of physiological data in hospital emergency departments

“…Algorithms are used to determine the reliability of waveform (e.g., electrocardiogram) and vital-sign data (e.g., heart rate) [6][7][8]. A primary focus of our research is early identification of patients with hemorrhage, and we have investigated a methodology involving multivariate classification [9] with the sequential probability ratio test [10] (the latter is an established technique for identifying abnormal patterns in a series of repeated measurements). The analysis interface has the ability to simultaneously run multiple instances of the analytic algorithms on the data from a given session at the same time, so that results from different algorithms can be compared.…”

Individualized performance prediction during total sleep deprivation: Accounting for trait vulnerability to sleep loss

“…Here, we use the minimum log-likelihood distance as our metric for determining an individual's phenotype. Alternate methods for determining the phenotype, such as the sequential probability ratio test [15], may provide rapid detection of phenotype and we intend to investigate their effectiveness as well.…”