Replications are important to science, but who will do them? One proposal is that students can conduct replications as part of their training. As a proof-of-concept for this idea, here we report a series of 11 pre-registered replications of findings from the 2015 volume of Psychological Science, all conducted as part of a graduate-level course.Congruent with larger, more systematic efforts, replications typically yielded smaller effects than originals: The modal outcome was partial support for the original claim.This work documents the challenges facing motivated students as they attempt to replicate previously published results on a first attempt. We describe the workflow and pedagogical methods that were used in the class and discuss implications both for the adoption of this pedagogical model and for replication research more broadly.Keywords: Replication; Reproducibility; Pedagogy; Experimental Methods REPLICATION THROUGH PEDAGOGY 3 Improving the Replicability of Psychological Science Through PedagogyReplicability is a core value for empirical research and there is increasing concern throughout psychology that more independent replication is necessary (Open Science Collaboration, 2015; Wagenmakers, Wetzels, Borsboom, Maas, & Kievit, 2012). Yet under the current incentive structure for science, replication is not typically valued for publication in top journals (Makel, Plucker, & Hegarty, 2012) or in metrics of research productivity (Koole & Lakens, 2012). One potential solution to this problem is to make replication an explicit part of pedagogy: that is, to teach students about experimental methods by asking them to run replication studies (Frank & Saxe, 2012; Grahe et al., 2012). Despite enthusiasm for this idea (Everett & Earp, 2015; M. King et al., 2016;LeBel, 2015;Standing, 2016), there is limited data beyond anecdotal reports and individual projects (Lakens, 2013; e.g., Phillips et al., 2015) to support its efficacy in producing wide-scale pedagogical adoption.In the current article, we describe the pedagogical and methodological approach to replication research taken in our graduate-level experimental methods course and address the practical barriers faced by instructors planning to incorporate replications into their courses. In our course, students conducted replications of published articles from the 2015 volume of the journal Psychological Science with rigorous instructor review at each major stage. The results of these replications are a microcosm of larger replication efforts, providing insight into both the difficulties of pedagogical replications and their promise as a method for improving the robustness of psychological research.We assess the challenges facing a student in choosing an article of interest and -in a single attempt, within constraints of budget, expertise, and effort -reproducing the findings. We consider a number of criteria for evaluating replication success, including statistical significance, effect size, a Bayesian measure of evidence (Etz & REPLICATION THROUG...

Communication is a cooperative endeavor that goes well beyond decoding sentences' literal meaning. Listeners actively construe the meaning of utterances from both their literal meanings and the pragmatic principles that govern communication. When communicators make pragmatically infelicitous statements, the effects can be similar to paltering-misleading speech that evokes false inferences from true statements. The American Diabetes Association's (ADA's) "Diabetes Myths" website provides a real-world case study in such misleading communications. Calling something a myth implies that it is clearly false. Instead, the ADA's "myths" are false only because of some technicality or uncharitable reading. We compared participants' baseline knowledge of diabetes with that of participants who read either the ADA's myths or the myths rewritten as questions that do not presuppose the statement is false. As predicted, exposure to the ADA's "myths," but not to the rephrased questions, reduced basic knowledge of diabetes. Our findings underscore the need to consider psycholinguistic principles in mass communications.

IntroductionFaculty are required to assess the development of residents using educational milestones. This descriptive study examined the end-of-rotation milestone-based evaluations of anesthesiology residents by rotation faculty directors. The goals were to measure: (1) how many of the 25 Accreditation Council for Graduate Medical Education (ACGME) anesthesiology subcompetency milestones were included in each of the residency’s rotations evaluations, (2) the percentage of evaluations sent to the rotation director that were actually completed by the director, (3) the length of time between the end of the residents' rotations and completion of the evaluations, (4) the frequency of straight line scoring, defined as the resident receiving the same milestone level score for all subcompetencies on the evaluation, and (5) how often a resident received a score below a Level 4 in at least one subcompetency in the three months prior to graduating.MethodsIn 2013, the directors for each the 24 anesthesia rotations in the Stanford University School of Medicine Anesthesiology Residency Program created new milestone-based evaluations to be used at the end of rotations to evaluate residents. The directors selected the subcompetencies from the list released by the ACGME that were most appropriate for their rotation. End-of-rotation evaluations for the post-graduate year (PGY)-2 to PGY-4 from July 1, 2014 to June 30, 2017 were retrospectively analyzed for a sample of 10 residents randomly selected from 22 residents in the graduating class.ResultsThe mean number of subcompetencies evaluated by each of the 24 rotations in the residency equaled 17.88 (standard deviation (SD): 3.39, range 10–24, median 18.5) from the available possible total of 25 subcompetencies. Three subcompetencies (medical knowledge, communication with patients and families, and coordination of patient care within the healthcare system) were included in the evaluation instruments of all 24 rotations. The three least frequently listed subcompetencies were: “acute, chronic, and cancer-related pain consultation/management” (25% of rotations had this on the end-of-rotation evaluation), “triage and management of critically ill patient in non-operative setting” (33%), and “education of patient, families, students, residents, and others” (38%). Overall, 418 end-of-rotation evaluations were issued and 341 (82%) completed, with 63% completed within one month, 22% between month one and two, and 15% after two months. The frequency of straight line scoring varied, from never occurring (0%) in three rotations to always occurring (100%) in two rotations, with an overall average of 51% (SD: 33%). Sixty-one percent of straight line scoring corresponded to the residents’ postgraduate year whereby, for example, a post-graduate year two resident received an ACGME Level 2 proficiency for all subcompetencies. Thirty-one percent of the straight line scoring was higher than the resident’s year of training (e.g., a PGY-2 received Level 3 or higher for all the subcompetencies). The rema...