The university participant pool is a key resource for behavioral research, and data quality is believed to vary over the course of the academic semester. This crowdsourced project examined time of semester variation in 10 known effects, 10 individual differences, and 3 data quality indicators over the course of the academic semester in 20 participant pools (N = 2,696) and with an online sample (N = 737). Weak time of semester effects were observed on data quality indicators, participant sex, and a few individual differences-conscientiousness, mood, and stress. However, there was little evidence for time of semester qualifying experimental or correlational effects. The generality of this evidence is unknown because only a subset of the tested effects demonstrated evidence for the original result in the whole sample. Mean characteristics of pool samples change slightly during the semester, but these data suggest that those changes are mostly irrelevant for detecting effects. Word count = 151Keywords: social psychology; cognitive psychology; replication; participant pool; individual differences; sampling effects; situational effects 4 Many Labs 3: Evaluating participant pool quality across the academic semester via replication University participant pools provide access to participants for a great deal of published behavioral research. The typical participant pool consists of undergraduates enrolled in introductory psychology courses that require students to complete some number of experiments over the course of the academic semester. Common variations might include using other courses to recruit participants or making study participation an option for extra credit rather than a pedagogical requirement. Research-intensive universities often have a highly organized participant pool with a participant management system for signing up for studies and assigning credit. Smaller or teaching-oriented institutions often have more informal participant pools that are organized ad hoc each semester or for an individual class.To avoid selection bias based on study content, most participant pools have procedures to avoid disclosing the content or purpose of individual studies during the sign-up process.However, students are usually free to choose the time during the semester that they sign up to complete the studies. This may introduce a selection bias in which data collection on different dates occurs with different kinds of participants, or in different situational circumstances (e.g., the carefree semester beginning versus the exam-stressed semester end).If participant characteristics differ across time during the academic semester, then the results of studies may be moderated by the time at which data collection occurs. Indeed, among behavioral researchers there are widespread intuitions, superstitions, and anecdotes about the "best" time to collect data in order to minimize error and maximize power. It is common, for example, to hear stories of an effect being obtained in the first part of the semester that then "d...

Many Labs 3 is a crowdsourced project that systematically evaluated time-of-semester effects across many participant pools. See the Wiki for a table of contents of files and to download the manuscript.

Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect ( p < .05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate the original effects. We revised the replication protocols and received formal peer review prior to conducting new replication studies. We administered the RP:P and revised protocols in multiple laboratories (median number of laboratories per original study = 6.5, range = 3–9; median total sample = 1,279.5, range = 276–3,512) for high-powered tests of each original finding with both protocols. Overall, following the preregistered analysis plan, we found that the revised protocols produced effect sizes similar to those of the RP:P protocols (Δ r = .002 or .014, depending on analytic approach). The median effect size for the revised protocols ( r = .05) was similar to that of the RP:P protocols ( r = .04) and the original RP:P replications ( r = .11), and smaller than that of the original studies ( r = .37). Analysis of the cumulative evidence across the original studies and the corresponding three replication attempts provided very precise estimates of the 10 tested effects and indicated that their effect sizes (median r = .07, range = .00–.15) were 78% smaller, on average, than the original effect sizes (median r = .37, range = .19–.50).

Very little is known about the oral health of, and access to, dental services among frail elders who live in the community and use an adult day health center (ADHC) for respite care. This pilot study evaluated the perceived oral health quality of life (OHQOL) of elders who used a mobile dental program in urban, suburban, and rural ADHC settings. Pre- and post-treatment interviews were conducted to evaluate OHQOL using the Geriatric Oral Health Assessment Index (GOHAI). ADHC records were used to obtain demographic, medical history and medication data. Following initial dental examinations and consent, dental treatment was provided at each ADHC. Of the 138 elders screened at three ADHCs, pre- and post-treatment data were obtained on 76 subjects following their treatment (mean four months later). The group's members were mostly female (64.5%) and Caucasian (71.6%). Their mean age was 76.8 (+/- 9.8), with an average of 12.4 teeth (34.2% edentulous); 67.7% were on Medicaid. On average they had 5.5 chronic diseases, hypertension being the most common (67.19%); 44.8% had a neurological disorder or dementia. GOHAI scores were generally high both pre- and post-treatment, reflecting high physical and psychosocial OHQOL and low levels of worry. GOHAI scores were correlated with chronic diseases; the more chronic diseases an individual had, the lower his or her total score pre- and post-treatment (r=-.24, r=-.26 respectively, p<.04). The more dental treatment needs an elder had, the lower his or her GOHAI (r=-.23, p<.05). Elders with more teeth reported higher GOHAI pre- and post-treatment (r=.36, r=.37 respectively, p<.002). Paired t-tests comparing pre- and post-treatment GOHAI scores revealed significant improvements in overall GOHAI (p<.001), and on two dimensions: physical (p<.005) and psychosocial (p<.002). The findings support the importance of providing on-site access to dental services in order to maintain the general OHQOL of frail elders, more specifically in the areas of physical and psychosocial well-being.

No abstract