Interrater reliability between in-person and telemedicine evaluations in obstructive sleep apnea

Yurcheshen, Michael; Pigeon, Wilfred R.; Marcus, C; Marcus, Jonathan A.; Messing, Susan; Nguyen, Kevin; Marsella, Jennifer

doi:10.5664/jcsm.9220

Cited by 10 publications

(5 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The area under the curve for all 4 conditions ranged between 0.62 and 0.69, indicating that search engine data are predictive of sleep disorders, with the prediction being of medium accuracy. We note that sleep disorders are known to be a difficult problem to assess, as evident, for example, in the low agreement among experts at different laboratories on the correct interpretation of diagnostic tests, such as polysomnography, in people with sleep disorders [ 21 ], and modest agreement rates between telemedicine and in-person diagnosis [ 22 ]. Moreover, our data show that the last 60 to 90 days of search data reach the highest prediction quality.…”

Section: Discussionmentioning

confidence: 99%

Identifying Sleep Disorders From Search Engine Activity: Combining User-Generated Data With a Clinically Validated Questionnaire

Cohen-Zion¹,

Gescheit²,

Levy³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

Background Sleep disorders are experienced by up to 40% of the population but their diagnosis is often delayed by the availability of specialists. Objective We propose the use of search engine activity in conjunction with a validated web-based sleep questionnaire to facilitate wide-scale screening of prevalent sleep disorders. Methods Search advertisements offering a web-based sleep disorder screening questionnaire were shown on the Bing search engine to individuals who indicated an interest in sleep disorders. People who clicked on the advertisements and completed the sleep questionnaire were identified as being at risk for 1 of 4 common sleep disorders. A machine learning algorithm was applied to previous search engine queries to predict their suspected sleep disorder, as identified by the questionnaire. Results A total of 397 users consented to participate in the study and completed the questionnaire. Of them, 132 had sufficient past query data for analysis. Our findings show that diurnal patterns of people with sleep disorders were shifted by 2-3 hours compared to those of the controls. Past query activity was predictive of sleep disorders, approaching an area under the receiver operating characteristic curve of 0.62-0.69, depending on the sleep disorder. Conclusions Targeted advertisements can be used as an initial screening tool for people with sleep disorders. However, search engine data are seemingly insufficient as a sole method for screening. Nevertheless, we believe that evaluable web-based information, easily collected and processed with little effort on part of the physician and with low burden on the individual, can assist in the diagnostic process and possibly drive people to seek sleep assessment and diagnosis earlier than they currently do.

show abstract

Section: Discussionmentioning

confidence: 99%

Identifying Sleep Disorders From Search Engine Activity: Combining User-Generated Data With a Clinically Validated Questionnaire

Cohen-Zion¹,

Gescheit²,

Levy³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…The results estimated that a sample size of thirty-five would be required to establish inter-rater and intra-rater reliability (H0 = 0.00, H1 = 0.70, α = 0.05, single tail, power = 0.95) [ 30 ]. To allow for attrition, we increased this estimated sample size by 10% and rounded up to the nearest whole number [ 31 , 32 ], leaving a sample size of thirty-nine.…”

Section: Methodsmentioning

confidence: 99%

Inter-Rater and Intra-Rater Reliability of Return-to-Work Screening Tests for UK Firefighters Following Injury

2022

View full text Add to dashboard Cite

The aim of this study was to assess the inter-rater and intra-rater reliability of a return-to-work (RTW) screening test to be used on UK firefighters following injury. The inter rater and intra-rater reliability of eight tasks involved in a screening test was used to assess readiness to RTW for UK firefighters following injury. These tasks included the following: (1) putting on and removing a breathing apparatus set (BA), (2) a ladder lift simulation, (3) a ladder carry simulation, (4) a light portable pump (LPP) lift and carry simulation, (5) a hose run, (6) a ladder climb with leg lock, (7) a casualty evacuation and (8) a confined space crawl simulation. The inter-rater reliability between each individual screening task was interpreted as Excellent (ICC = 0.94–1.00) for eleven (68.75%) of the screening task videos and as Good (ICC = 0.75–0.88) for five (31.25%) of the screening task videos. Intra-rater reliability was interpreted as Excellent (ICC = 1) for twenty-six participants (74.3%), Good (ICC = 0.76–0.88) for eight participants (22.9%) and Moderate for one participant (2.8%). Due to the reliability of this screening test, it allows conclusions to be made from the results which can inform a RTW decision for a firefighter.

show abstract

“…Nevertheless, these broader concerns regarding OSA pretest probability determination in no way diminish the important work of Yurcheshen et al 6 Their research highlights how increased utilization of sleep telemedicine forces us all to question both how and why we do what we do. Suboptimal interrater reliability, whether assessed through telemedicine or in-person care, is not cause to eliminate a particular physical examination maneuver or to avoid the patient encounter modality (eg, telemedicine) in which it was performed.…”

mentioning

confidence: 99%

“…Yurcheshen et al 6 examine these questions in this issue of the Journal of Clinical Sleep Medicine. One board-certified sleep medicine physician interviewed and examined a patient in the clinic while a different board-certified sleep medicine physician completed a similar evaluation with the same patient through telemedicine.…”

mentioning

confidence: 99%

“…The American Academy of Sleep Medicine's 2015 position paper defines it as "A legal patient/clinician encounter using electronic communication." 8 Yurcheshen et al 6 utilized a form of telemedicine called clinical video telehealth (CVT), which implies real-time (synchronous) audiovisual communication occurred between patient and practitioner. CVT is subdivided into center-to-center (C2C) telemedicine where both patient and practitioner are in clinical settings, center-to-home (C2H) telemedicine where the patient is in a nonclinical setting, and out-of-center (OOC) telemedicine where both patient and practitioner are in nonclinical settings.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Exploring telemedicine evaluation reliability: ahead of its time and long overdue

Fields

2021

Journal of Clinical Sleep Medicine

View full text Add to dashboard Cite

Interrater reliability between in-person and telemedicine evaluations in obstructive sleep apnea

Cited by 10 publications

References 28 publications

Identifying Sleep Disorders From Search Engine Activity: Combining User-Generated Data With a Clinically Validated Questionnaire

Identifying Sleep Disorders From Search Engine Activity: Combining User-Generated Data With a Clinically Validated Questionnaire

Inter-Rater and Intra-Rater Reliability of Return-to-Work Screening Tests for UK Firefighters Following Injury

Exploring telemedicine evaluation reliability: ahead of its time and long overdue

Contact Info

Product

Resources

About