The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative characteristics and estimates of quality and rigor that may guide for future studies are lacking. To address this gap, we conducted a pooled analysis of 69 concept mapping studies to describe characteristics across study phases, generate specific indicators of validity and reliability, and examine the relationship between select study characteristics and quality indicators. Individual study characteristics and estimates were pooled and quantitatively summarized, describing the distribution, variation and parameters for each. In addition, variation in the concept mapping data collection in relation to characteristics and estimates was examined. Overall, results suggest concept mapping yields strong internal representational validity and very strong sorting and rating reliability estimates. Validity and reliability were consistently high despite variation in participation and task completion percentages across data collection modes. The implications of these findings as a practical reference to assess the quality and rigor for future concept mapping studies are discussed.

Evaluative bibliometrics uses advanced techniques to assess the impact of scholarly work in the context of other scientific work and usually compares the relative scientific contributions of research groups or institutions. Using publications from the National Institute of Allergy and Infectious Diseases (NIAID) HIV/AIDS extramural clinical trials networks, we assessed the presence, performance, and impact of papers published in 2006–2008. Through this approach, we sought to expand traditional bibliometric analyses beyond citation counts to include normative comparisons across journals and fields, visualization of co-authorship across the networks, and assess the inclusion of publications in reviews and syntheses. Specifically, we examined the research output of the networks in terms of the a) presence of papers in the scientific journal hierarchy ranked on the basis of journal influence measures, b) performance of publications on traditional bibliometric measures, and c) impact of publications in comparisons with similar publications worldwide, adjusted for journals and fields. We also examined collaboration and interdisciplinarity across the initiative, through network analysis and modeling of co-authorship patterns. Finally, we explored the uptake of network produced publications in research reviews and syntheses. Overall, the results suggest the networks are producing highly recognized work, engaging in extensive interdisciplinary collaborations, and having an impact across several areas of HIV-related science. The strengths and limitations of the approach for evaluation and monitoring research initiatives are discussed.

The health promoting schools concept reflects a comprehensive and integrated philosophy to improving student and personnel health and well-being. Conceptualized as a configuration of interacting, interdependent parts connected through a web of relationships that form a whole greater than the sum of its parts, school health promotion initiatives often target several levels (e.g. individual, professional, procedural and policy) simultaneously. Health promoting initiatives, such as those operationalized under the whole school approach, include several interconnected components that are coordinated to improve health outcomes in complex settings. These complex systems interventions are embedded in intricate arrangements of physical, biological, ecological, social, political and organizational relationships. Systems thinking and characteristics of complex adaptive systems are introduced in this article to provide a perspective that emphasizes the patterns of inter-relationships associated with the nonlinear, dynamic and adaptive nature of complex hierarchical systems. Four systems thinking areas: knowledge, networks, models and organizing are explored as a means to further manage the complex nature of the development and sustainability of health promoting schools. Applying systems thinking and insights about complex adaptive systems can illuminate how to address challenges found in settings with both complicated (i.e. multi-level and multisite) and complex aspects (i.e. synergistic processes and emergent outcomes).