Range Based Confusion Matrix for Imbalanced Time Series Classification

Zhou, Xianzhe; Valle, Arturo Del

doi:10.1109/cdma47397.2020.00006

Cited by 12 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…True negatives are more difficult to determine. Even though Zhou and Del Valle [32] recommend treating all the rest as true negatives, that approach does not hold in our case, as there is no predetermined number of change events that can happen during a given time series. The maximum number of events given a one-year spacing interval for the three years of reference data would be seven (four predicted breaks and three reference points of change).…”

Section: Reference Data and Validationmentioning

confidence: 97%

“…For the purpose of assessing how well time series break predictions match the reference data, we developed a distance-in-time approach similar to that proposed by Zhou and Del Valle [32]. We treated each reference and predicted break as an event and then determined whether it was a true positive, false positive, false negative or true negative.…”

Section: Reference Data and Validationmentioning

confidence: 99%

See 1 more Smart Citation

BFAST Lite: A Lightweight Break Detection Method for Time Series Analysis

et al. 2021

View full text Add to dashboard Cite

BFAST Lite is a newly proposed unsupervised time series change detection algorithm that is derived from the original BFAST (Breaks for Additive Season and Trend) algorithm, focusing on improvements to speed and flexibility. The goal of the BFAST Lite algorithm is to aid the upscaling of BFAST for global land cover change detection. In this paper, we introduce and describe the algorithm and then compare its accuracy, speed and features with other algorithms in the BFAST family: BFAST and BFAST Monitor. We tested the three algorithms on an eleven-year-long time series of MODIS imagery, using a global reference dataset with over 30,000 point locations of land cover change to validate the results. We set the parameters of all algorithms to comparable values and analysed the algorithm accuracy over a range of time series ordered by the certainty of that the input time series has at least one abrupt break. To compare the algorithm accuracy, we analysed the time difference between the detected breaks and the reference data to obtain a confusion matrix and derived statistics from it. Lastly, we compared the processing speed of the algorithms using both the original R code as well as an optimised C++ implementation for each algorithm. The results showed that BFAST Lite has similar accuracy to BFAST but is significantly faster, more flexible and can handle missing values. Its ability to use alternative information criteria to select the number of breaks resulted in the best balance between the user’s and producer’s accuracy of detected changes of all the tested algorithms. Therefore, BFAST Lite is a useful addition to the BFAST family of unsupervised time series break detection algorithms, which can be used as an aid in narrowing down areas with changes for updating land cover maps, detecting disturbances or estimating magnitudes and rates of change over large areas.

show abstract

Section: Reference Data and Validationmentioning

confidence: 97%

Section: Reference Data and Validationmentioning

confidence: 99%

BFAST Lite: A Lightweight Break Detection Method for Time Series Analysis

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The model's effectiveness was measured using a confusion matrix, accuracy (CA), precision, recall, and F1-score (F1) [33]. The confusion matrix depicts the present state of the dataset as well as the number of accurate and wrong model predictions [34]. The proportion of accurate predictions to all predictions is measured by accuracy, which is a crucial and intuitive metric.…”

Section: Performance Evaluation Of the Modelmentioning

confidence: 99%

An LSTM-based prediction model for gradient-descending optimization in virtual learning environments

Ismanto,

Effendi

2024

Comput Sci Inf Technol

View full text Add to dashboard Cite

A virtual learning environment (VLE) is an online learning platform that allows many students, even millions, to study according to their interests without being limited by space and time. Online learning environments have many benefits, but they also have some drawbacks, such as high dropout rates, low engagement, and students' self-regulated behavior. Evaluating and analyzing the students' data generated from online learning platforms can help instructors to understand and monitor students learning progress. In this study, we suggest a predictive model for assessing student success in online learning. We investigate the effect of hyperparameters on the prediction of student learning outcomes in VLEs by the long short-term memory (LSTM) model. A hyperparameter is a parameter that has an impact on prediction results. Two optimization algorithms, adaptive moment estimation (Adam) and Nesterov-accelerated adaptive moment estimation (Nadam), were used to modify the LSTM model's hyperparameters. Based on the findings of research done on the optimization of the LSTM model using the Adam and Nadam algorithm. The average accuracy of the LSTM model using Nadam optimization is 89%, with a maximum accuracy of 93%. The LSTM model with Nadam optimisation performs better than the model with Adam optimisation when predicting students in online learning.

show abstract

“…But for hazard prediction, it is desirable that a monitor generates alerts before a hazard happens. So we adopt a modified version of standard classification metrics [71], proposed for sequential data [72] [73] [74], where a tolerance window before the start time of hazard (t h ) is used for calculation of the metrics (see Fig. 6).…”

Section: Metricsmentioning

confidence: 99%

Data-driven Design of Context-aware Monitors for Hazard Prediction in Artificial Pancreas Systems

Xugui¹,

Ahmed²,

Aylor³

et al. 2021

Preprint

View full text Add to dashboard Cite

Medical Cyber-physical Systems (MCPS) are vulnerable to accidental or malicious faults that can target their controllers and cause safety hazards and harm to patients. This paper proposes a combined model and data-driven approach for designing context-aware monitors that can detect early signs of hazards and mitigate them in MCPS. We present a framework for formal specification of unsafe system context using Signal Temporal Logic (STL) combined with an optimization method for patient-specific refinement of STL formulas based on real or simulated faulty data from the closed-loop system for the generation of monitor logic. We evaluate our approach in simulation using two state-of-the-art closed-loop Artificial Pancreas Systems (APS). The results show the context-aware monitor achieves up to 1.4 times increase in average hazard prediction accuracy (F1score) over several baseline monitors, reduces false-positive and false-negative rates, and enables hazard mitigation with a 54% success rate while decreasing the average risk for patients.

show abstract

Range Based Confusion Matrix for Imbalanced Time Series Classification

Cited by 12 publications

References 16 publications

BFAST Lite: A Lightweight Break Detection Method for Time Series Analysis

BFAST Lite: A Lightweight Break Detection Method for Time Series Analysis

An LSTM-based prediction model for gradient-descending optimization in virtual learning environments

Data-driven Design of Context-aware Monitors for Hazard Prediction in Artificial Pancreas Systems

Contact Info

Product

Resources

About