Search citation statements
Paper Sections
Citation Types
Year Published
Publication Types
Relationship
Authors
Journals
BACKGROUND Machine learning models that exploit rich digital phenotyping data to forecast mental states could improve clinical practice in psychiatry. A current limitation is that such predictive models tend to disregard a frequent property of the prediction targets, namely the ordinal nature of the rating scales they are often obtained from. OBJECTIVE We first aimed to contrast the performances of ordinal regression vs. binary classification to predict various mental states for different forecast horizons. We also assessed how the tree-based eXtreme Gradient Boosting (XGBoost) algorithm performs compared to the long short-term memory (LSTM) algorithm, a popular type of recurrent neural network in digital phenotyping studies aimed at forecasting mental states. METHODS The CrossCheck dataset includes self-reports of mental states and smartphone sensor data contributed by patients with schizophrenia. Participants completed surveys on various mental states every 2-3 days on 4-point ordinal rating scales. Passive sensing data was collected continuously and aggregated over 6-hour periods. We trained 120 machine learning models to forecast mental states from passive sensing data: 10 mental states (e.g., Calm, Depressed, Seeing things) on 2 predictive tasks (ordinal regression, binary classification) with 2 learning algorithms (XGBoost, LSTM) over 3 forecast horizons (same day, next day, next week). While models were primarily evaluated with performance metrics that account for class imbalance (macro-averaged mean absolute error -MAMAE- for ordinal regression and balanced accuracy -BAcc- for binary classification), the impact of using metrics that do not deal with imbalance (mean absolute error, accuracy) was also investigated. RESULTS The dataset included 6364 surveys and 23,551 days of smartphone data from 62 participants. Marked class imbalance was observed for the ordinal labels, an issue that was only partially resolved by recoding original labels into binary classes. Globally, 45/60 ordinal regression models performed significatively above baseline with MAMAE between 1.19 and 0.77, and 58/60 binary classification models were significant with BAcc between 58% and 73%. Of note, evaluation metrics that do not deal with class imbalance erroneously reflected good performance. After scaling performance metrics to allow their comparison, ordinal regression and binary classification models achieved comparable performance on average. XGBoost models performed better or on par with LSTM models. As the forecast horizon expanded, a significant yet very small decrease in performance was observed. CONCLUSIONS The targets of mental state forecast models should preserve the valuable clinical information contained in ordinal rating scales. This is especially true given recoding multiple ordinal classes into binary classes does not lead to any gain in predictive performance. Moreover, model development should account for class imbalance, particularly so for ordinal regression where imbalance across classes is often more pronounced. Finally, our findings do not lend support to the implicitly assumed superiority of recurrent neural networks for forecasting.
We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
334 Leonard St
Brooklyn, NY 11211
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.