Differentiation of central disorders of hypersomnolence with manual and artificial-intelligence-derived polysomnographic measures

Abstract

Differentiation of central disorders of hypersomnolence (DOH) is challenging but important for patient care. This study aimed to investigate whether biomarkers derived from sleep structure evaluated both by manual scoring as well as with artificial intelligence (AI) algorithms allow distinction of patients with different DOH. We included video-polysomnography data of 40 narcolepsy type 1 (NT1), 26 narcolepsy type 2 (NT2), 23 idiopathic hypersomnia (IH) patients and 54 subjects with subjective excessive daytime sleepiness (sEDS). Sleep experts manually scored sleep stages. A previously validated AI algorithm was employed to obtain automatic hypnograms and hypnodensity graphs (where each epoch is represented as a mixture of sleep stage probabilities). One-thousand-three features describing sleep architecture and instability were extracted from manual/automatic hypnogram and hypnodensity graphs. After feature selection, random forest classifiers were trained and tested in a 5-fold-cross-validation scheme to distinguish groups pairwise (NT1-vs-NT2, NT1-vs-IH, …) and single groups from the pooled remaining ones (NT1-vs-rest, NT2-vs-rest,…). The accuracy/F1-score values obtained in the test sets were: 0.74±0.04/0.79±0.05 (NT1-vs-NT2), 0.89±0.09/0.91±0.08 (NT1-vs-IH), 0.93±0.06/0.91±0.07 (NT1-vs-sEDS), 0.88±0.04/0.80±0.07 (NT1-vs-rest), 0.65±0.10/0.70±0.09 (NT2-vs-IH), 0.72±0.12/0.60±0.10 (NT2-vs-sEDS), 0.54±0.19/0.38±0.13 (NT2-vs-rest), 0.57±0.11/0.35±0.18 (IH-vs-sEDS), 0.71±0.08/0.35±0.10 (IH-vs-rest) and 0.76±0.08/0.71±0.13 (sEDS-vs-rest). The results confirm previous findings on sleep instability in NT1 patients and show that combining manual and automatic AI-based sleep analysis could be useful for better distinction of NT2 from IH, but no precise sleep biomarker of NT2 or IH could be identified. Validation in a larger and multi-centric cohort is needed to confirm these findings.

Overview of the hypnodensity approach to scoring sleep for polysomnography and home sleep testing

“…Table 2 summarizes the kappa values for the 20 validation studies of the 6 AI-based autoscoring algorithms outputting the hypnodensity graph (Stephansen et al, 2018;Cesari et al, 2021Cesari et al, , 2022Vallat and Walker, 2021;Anderer et al, 2022b;Brandmayr et al, 2022;Bakker et al, 2023;Fiorillo et al, 2023b). As can be seen in Table 2, Cohen's kappa for the 5-stage comparison was comparable between the six algorithms.…”

“…Stephansen et al (2018), Bakker et al (2023) and Fiorillo et al (2023b) validated their algorithms in the same IS-RC cohort and reported, as compared to the consensus of 6 scorers, kappa values between 0.67 and 0.78. Cesari et al (2022) compared the Stanford-STAGES and YASA algorithm in a dataset of patients with central disorders of hypersomnolence and reported almost identical kappa values for the two algorithms (0.747 and 0.755 for Stanford-STAGES and YASA, respectively). These findings suggest that modern AI-based autoscoring systems offer valid alternatives to manual expert scoring and that the role of manual adjustment and expert review of automatic scorings might no longer be required.…”

“…Based on these probabilities, it is possible to create a hypnodensity chart from autoscoring which can be directly compared to the hypnodensity chart based on multiple expert scorings. In Table 2 we summarize publications using AI-algorithms for sleep staging and reporting autoscored hypnodensity graphs (Stephansen et al, 2018;Cesari et al, 2021Cesari et al, , 2022Vallat and Walker, 2021;Anderer et al, 2022b;Brandmayr et al, 2022;Bakker et al, 2023;Fiorillo et al, 2023b). In addition to information regarding the training datasets, the epoch encoder including feature extraction, the sequence encoder and classifier, and the test datasets, the Cohen's kappa values obtained in each study are given.…”

“…Table 3 summarizes validation results of AI-algorithms that applied a hold-out or crossvalidation; i.e., an internal validation based on data from the same dataset that has been used for training (Supratak et al, 2017;Sors et al, 2018;Phan et al, 2019;Zhang et al, 2019;Abou Jaoude et al, 2020;Guillot et al, 2020;Korkalainen et al, 2020;Sun et al, 2020a;Alvarez-Estevez and Rijsman, 2021;Fiorillo et al, 2021Fiorillo et al, , 2023bJia et al, 2021;Nasiri and Clifford, 2021;Olesen et al, 2021;Pathak et al, 2021;Vallat and Walker, 2021;Brandmayr et al, 2022;Cho et al, 2022;Ji et al, 2022;Li C. et al, 2022;Sharma et al, 2022;Yubo et al, 2022). Table 4 summarizes the validation results of AI-algorithms which have been validated in datasets completely unseen by the model (Anderer et al, 2018(Anderer et al, , 2022bBiswal et al, 2018;Patanaik et al, 2018;Stephansen et al, 2018;Zhang et al, 2019;Abou Jaoude et al, 2020;Alvarez-Estevez and Rijsman, 2021;Cesari et al, 2021Cesari et al, , 2022Vallat and Walker, 2021;Bakker et al, 2023). The reported Cohen's kappa values ranged between 0.60 [external validation in 70 patients with Parkinson's disease (Patanaik et al, 2018)] to 0.91 [internal 20-fold epochwise cross validatio...…”

“…By means of a Gaussian predictor classifier they achieved a specificity of 96% and a sensitivity of 91% for classifying narcolepsy type-1 as validated in independent datasets. In a more recent study, Cesari et al (2022) investigated whether biomarkers describing sleep instability and architecture derived from both manual hypnogram and automatic hypnogram and hypnodensity graphs might differentiate between distinct disorders of hypersomnolence, such as narcolepsy type-1, narcolepsy type-2, idiopathic hypersomnia and subjective excessive daytime sleepiness. They extracted features from manual and automatic hypnograms, such as standard features, transition features, bouts features, features describing stability and fragmentation of sleep stages as well as the distribution of sleep stages across the night, and REM sleep-specific features such as the number of nightly sleep onset REM periods.…”