Working with error and uncertainty to increase measurement validity

Amrein‐Beardsley, Audrey; Barnett, Joshua H.

doi:10.1007/s11092-012-9146-6

Cited by 9 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…17 Consequently, no single method can assess the value of training programs, and more than one method is used for the assessment of disaster medicine training programs. 17,21 The course was evaluated by different tools for multiple components: pre-and post-tests of knowledge and skills, a post-test on intervention for sieve triage, and pre-and post-self-assessment of perceived knowledge and skills related to performance objectives of prehospital components of a MACSIM simulation exercise. Assessment of the gain in knowledge from the course by comparing pre-and post-test results showed a statistically significant enhancement in knowledge (P = .003).…”

Section: Discussionmentioning

confidence: 99%

Course in Prehospital Major Incidents Management for Health Care Providers in Saudi Arabia

Bajow

Alassaf

Cluntun

2018

Prehosp. Disaster med.

View full text Add to dashboard Cite

IntroductionUnacceptable practices of health care providers during disasters have been observed because they work outside the scope of their daily practices and have inadequate training. A greater need for the involvement of health professionals in disaster management has been noted in Saudi Arabia. This study evaluates the efficacy of a training course in prehospital major incident management for health care providers in Saudi Arabia.MethodsAn interactive course for general principles in prehospital major incident management was developed with domains and core competencies. The course was designed according to the local context and was based on international standards. It was piloted over four days at the Officers Club of the Ministry of Interior (Riyadh, Saudi Arabia) and was sponsored by Mohammed Bin Naif Medical Center, King Fahd Security College in Riyadh, Saudi Arabia. The participants (n=29) were from different disciplines from main government health facilities in Riyadh. They completed a pre-test and a post-test.ResultsThe overall score was 55.1% on the pre-test and 68.4% on the post-test (Wilcoxon test for paired samples, P <.05). Three out of the four domains had significant difference between pre- and post-test results, as well as the overall total knowledge.Conclusion:Conducting inter-disciplinary and competency-based disaster medicine courses for health care providers can augment appropriate disaster preparedness for major incidents in Saudi Arabia.BajowNA,AlAssafWI,CluntunAA.Course in prehospital major incidents management for health care providers in Saudi Arabia.Prehosp Disaster Med.2018;33(6):587–595.

show abstract

Section: Discussionmentioning

confidence: 99%

Course in Prehospital Major Incidents Management for Health Care Providers in Saudi Arabia

Bajow

Alassaf

Cluntun

2018

Prehosp. Disaster med.

View full text Add to dashboard Cite

show abstract

“…Under COVID-19 conditions, Kane's Validation Model challenges the intended uses of current test scores. Amrein-Beardsley and Barnett (2012) urged education policymakers to move towards a holistic measurement system to address the limitations and utility of standardized tests.…”

Section: Conceptual Frameworkmentioning

confidence: 99%

Perceptions of Test Score Pollution Stemming From COVID-19 and State Testing: An Exploratory Case Study

Kalemdaroglu-Wheeler

2023

AM J QUALITATIVE RES

View full text Add to dashboard Cite

The purpose of this qualitative exploratory case study was to explore teachers' and administrators' perceptions of test score pollution deriving from COVID-19-related issues that may affect students' test scores on state-mandated standardized tests for grades six through 12 in a state along the Atlantic Coast of the United States. Four research questions were devised to investigate participants' perceptions of factors stemming from COVID-19-related issues that may alter students' performance on state standardized tests, commonly referred to as test score pollution. The conceptual framework centered around Heisenberg's Uncertainty Principle, Campbell's Law, and Kane's Validation Model. A purposeful stratified sampling method was utilized for participant inclusion criteria. The study sample included four middle school teachers in grades six through eight, two high school teachers, two middle school administrators, and two high school administrators. The data collection method included semi-structured interviews. Data analysis was conducted in unison with memoing, member checking, In Vivo, Descriptive, Focused, Axial, and Process Coding methods. The following themes emerged through data analysis: (a) inadequately preparing students for state standardized tests, (b) de-valuing of education, (c) understanding students' emotional well-being, (d) providing data-driven support, (e) questioning test validity, and (f) recognizing magnified disparities among students.

show abstract

“…Finally, the present paper addressed only methodological questions pertaining to VA modeling. As pointed out by two anonymous reviewers of our paper, other essential aspects of VA models and their application require further investigation (see, for example, the works by Amrein-Beardsley and Barnett 2012;Everson 2017). This involves the following questions: In which countries are VA scores used to make high-stakes decisions (e.g., personnel decisions)?…”

Section: Limitationsmentioning

confidence: 99%

Methodological issues in value-added modeling: an international review from 26 countries

et al. 2019

View full text Add to dashboard Cite

Value-added (VA) modeling can be used to quantify teacher and school effectiveness by estimating the effect of pedagogical actions on students' achievement. It is gaining increasing importance in educational evaluation, teacher accountability, and high-stakes decisions. We analyzed 370 empirical studies on VA modeling, focusing on modeling and methodological issues to identify key factors for improvement. The studies stemmed from 26 countries (68% from the USA). Most studies applied linear regression or multilevel models. Most studies (i.e., 85%) included prior achievement as a covariate, but only 2% included noncognitive predictors of achievement (e.g., personality or affective student variables). Fifty-five percent of the studies did not apply statistical adjustments (e.g., shrinkage) to increase precision in effectiveness estimates, and 88% included no model diagnostics. We conclude that research on VA modeling can be significantly enhanced regarding the inclusion of covariates, model adjustment and diagnostics, and the clarity and transparency of reporting.Keywords Value-added modeling . Literature review . Primary and secondary education . Teacher effectiveness . School effectiveness What is the added value from attending a certain school or being taught by a certain teacher? To answer this question, the value-added (VA) model was developed. In this model, the actual achievement attained by students attending a certain school or being

show abstract

Working with error and uncertainty to increase measurement validity

Cited by 9 publications

References 24 publications

Course in Prehospital Major Incidents Management for Health Care Providers in Saudi Arabia

Course in Prehospital Major Incidents Management for Health Care Providers in Saudi Arabia

Perceptions of Test Score Pollution Stemming From COVID-19 and State Testing: An Exploratory Case Study

Methodological issues in value-added modeling: an international review from 26 countries

Contact Info

Product

Resources

About