Inter-rater Reliability of the Ramsay Sedation Scale for Critically-ill Intubated Patients

Abstract

Introduction: Titratable palliative sedation (TPS) is frequently applied in sedative therapeutics to ameliorate unendurable and refractory distress via reduction in patient consciousness. TPS may be adjusted based on objective and subjective data: vitals, labs, Ramsay Sedation Scale (RSS), and Richmond Agitation-Sedation Scale (RASS). Inappropriate dosing, including over-sedation from variability in clinician assessment of sedation scales, can contribute to significant negative clinical outcomes. We evaluated inter-rater reliability (IRR) and its relationship to variations in dosing to determine whether additional training in sedation scale assessment is necessary at our community institution.Methods: This was a prospective study assessing sedation in intensive care unit (ICU) mechanically ventilated patients without neurogenic abnormalities. Non-nursing healthcare personnel conducted independent sedation assessments using the RSS and compared their evaluations to those documented by the nursing staff. Data obtained from the patients' chart included: demographics, Ramsay Score, past medical history, diagnosis, and body mass index (BMI). Post-analysis, non-nursing healthcare personnel scores were classified into three categories: equal to, higher than, or lower than those charted by nursing staff.Results: There were 83 random RSS assessments conducted in 44 patients with a mean age of 63.6 +/10.09 years (range: 38-82) and a mean BMI of 31.2 +/12.4 (range: 15-77). 19/42 (45%) patients had a diagnosis of respiratory failure or pneumonia. Other diagnoses included congestive heart failure (3), seizures (5), aortic valve replacement (1), small bowel obstruction (1), drug overdose (2), cardiac arrest (2), and urinary tract infection (1), ST-elevated myocardial infarction (2), pulmonary embolism (2), coronary artery bypass graft (1), sepsis (1), hemoptysis (1), altered mental status (1). Non-nursing healthcare professionals' assessments were compared to nurses' and observed to be equal in 29%, higher in 59%, and lower in 12% of the cases. Of the 83 assessments, the average RSS score non-nursing healthcare professionals assigned was 4.8 +/1.6 while the nurses' charted average was3.39 +/- 0.97; a mean difference of 1.45, 95% CI (1.04 - 1.85)p< 0.0001.Conclusions: Our data demonstrated equal RSS ratings in only 29% of cases for non-nursing healthcare personnel and nurses’ evaluations. Without proper education, the RSS may not be a reliable tool for sedation assessments and may result in over-sedation of critically ill patients. Recurrent nursing education is warranted to ensure proper use and optimization of the RSS.

A systematic review of quality improvement initiatives for continuous sedation until death

“…Moderate to very high inter-rater reliability was found for the VICS, RASS, KNMG scale, and CSPC. However, the study of Deol 38 indicated a disagreement amongst healthcare professionals of various disciplines when using the RSS to assess sedation in critically ill patients in intensive care units, with equal RSS ratings by nurses and non-nursing personnel in only 29% of cases. The study of Arevalo found discriminative and evaluative validity for RASS and the KNMG scale (ρ = 0.836), the study of Benitez-Rosario found a strong correlation of the RASS and the RSS (Spearman’s p , −0.89; p < 0.001) and GCS (Spearman’s p, 0.85; p < 0.001) and the study of Conçalves found a very high correlation of the CSPC to the VAS (Spearman’s p , 0.94, p < 0.001) and GCS (Spearman’s p , −0.82, p < 0.001).…”

“…Use of observer based sedation scales (n = 5). The first group focused on the use of observer-based sedation scales and testing their reliability and validity in a palliative care setting and covered the "Minnesota Sedation Assessment Tool" (MSAT) 34 , "Richmond Agitation-Sedation Scale" (RASS) [34][35][36] , "Vancouver Interaction and Calmness Scale" (VICS) 34 , a sedation score proposed in the Guideline for Palliative Sedation of the Royal Dutch Medical Association (KNMG) 34 , the "Consciousness Scale for Palliative Care" (CSPC) 37 and the Ramsay Sedation Scale (RSS) 38 . The usefulness of the scales was assessed in four of the five studies and this shows that the RASS [34][35][36] and CSPC 37 are considered to be very useful, easy to use, not time consuming and can be used with minimal training.…”

“…Moderate to very high interrater reliability was found for the VICS, RASS, KNMG scale, and CSPC. However, the study of Deol 38 indicated a disagreement amongst healthcare professionals of various disciplines when using the RSS to assess sedation in critically ill patients in intensive care units, with equal RSS ratings by nurses and non-nursing personnel in only 29% of cases. The study of Arevalo found discriminative and evaluative validity for RASS and the KNMG scale (ρ = 0.836), the study of Benitez-Rosario found a strong correlation of the RASS and the RSS (Spearman's p, −0.89; p < 0.001) and GCS (Spearman's p, 0.85; p < 0.001) and the study of Conçalves found a very high correlation of ("deep sedation"/exp OR "deep sedation":ti,ab,kw OR "palliative sedation":ti,ab,kw OR "terminal sedation":ti,ab,kw OR "continuous sedation":ti,ab,kw OR "sedated":ti,ab,kw) (embase.com) AND ("terminal care"/exp OR "terminal care":ti,ab,kw OR "palliative therapy"/exp OR "palliative therapy":ti,ab,kw OR "terminally ill patient"/exp OR "terminally ill patient":ti,ab,kw OR "end of life":ti,ab,kw OR "incurable":ti,ab,kw OR "palliative":ti,ab,kw OR "terminal*":ti,ab,kw OR "life- ("deep sedation" OR (MM "Sedation") OR "palliative sedation" OR "terminal sedation" OR "continuous sedation" OR "sedated") (EBSCOhost) AND ((MM "Terminal Care") OR "Terminal Care" OR (MM "Palliative Care") OR "Palliative Care" OR (MM "Terminally Ill Patients") OR "terminally ill" OR "end of life" OR "incurable" OR "palliative" OR "Terminal" OR "terminally" OR (MM "Critical Illness") OR "life-threatening")) Web of Science Core Collection (TS=("deep sedation" OR "palliative sedation" OR "terminal sedation" "continuous sedation" OR "sedated")) AND (TS=("terminal care" OR "palliative care" OR "terminally ill" OR "end-of-life" OR "incurable" OR "palliative" OR "terminal*" OR "life-threatening"))…”

Sedación paliativa en enfermedad terminal

“…La monitorización del nivel de conciencia y la respuesta a estímulos juega un papel importante para definir y dirigir una administración adecuada de medicamentos. La RSS es uno de los instrumentos clínicos más usados para la monitorización de la sedación en el contexto de la SP (36).…”

Sedation and Delirium in the Intensive Care Unit—A Practice-Based Approach

“…Pain in adult ICU patients should be treated before a sedative agent is considered and the management of pain should be guided by routine pain assessment (analgesia-first sedation or analgesia-based sedation) NA (Good practice statement) NA Use a stepwise approach for pain and sedation management that is protocol-based and assessment-driven Despite being widely accepted as the gold standard in current sedation monitoring, the subjective nature of RASS and SAS assessments predisposed both scales to variability and uncertainty on the exact level of sedation that is needed at any point in time; with lower RASS scores, variable inter-rater reliability becomes particularly glaring in different institutions. [10][11][12]…”